00000800:00000200:0.1F:1775762860.949224:0:2401:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000160a4f2b] -> 12345-192.168.202.110@tcp (2)
00000800:00000200:3.1F:1775762860.960602:0:2390:0:(socklnd.c:384:ksocknal_incr_conn_count()) Add conn type 1, ksnr_connected 2 ksnr_max_conns 1
00000800:00000200:3.1:1775762860.960621:0:2390:0:(socklnd_cb.c:748:ksocknal_queue_tx_locked()) Sending to 12345-192.168.202.110@tcp ip 192.168.202.110:988
00000800:00000200:3.1:1775762860.960626:0:2390:0:(socklnd_cb.c:767:ksocknal_queue_tx_locked()) Packet 00000000da9d6faf type 2, nob 96 niov 1 nkiov 0
00000800:00000200:3.1:1775762860.967091:0:2390:0:(socklnd.c:384:ksocknal_incr_conn_count()) Add conn type 2, ksnr_connected 6 ksnr_max_conns 1
00000800:00000200:1.1F:1775762860.977989:0:2390:0:(socklnd.c:384:ksocknal_incr_conn_count()) Add conn type 3, ksnr_connected e ksnr_max_conns 1
00010000:00010000:1.0:1775764822.264960:0:70919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f1498178800/0x23cab7dad48e5f75 lrc: 2/1,0 mode: PR/PR res: [0x200000405:0x75:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x929304787232a39 expref: -99 pid: 70919 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0
00010000:00010000:1.0:1775764822.264974:0:70919:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f1498178800/0x23cab7dad48e5f75 lrc: 2/1,0 mode: PR/PR res: [0x200000405:0x75:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x929304787232a39 expref: -99 pid: 70919 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0
00000080:00200000:1.0:1775764822.264978:0:70919:0:(namei.c:953:ll_intent_lock()) intent lock 8 on i1 [0x200000405:0x75:0x0] suppgids 0 0: rc 1
00000080:00010000:1.0:1775764822.264984:0:70919:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000405:0x75:0x0](ffff9f14a10b63c8)
00000080:00010000:1.0:1775764822.264986:0:70919:0:(llite_internal.h:1984:ll_set_lock_data()) setting l_data to inode [0x200000405:0x75:0x0](ffff9f14a10b63c8) for lock 0x23cab7dad48e5f75
00000080:00010000:1.0:1775764822.264991:0:70919:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x23cab7dad48e5f75 from it ffffb06c0150bcd8
00010000:00010000:1.0:1775764822.264993:0:70919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f1498178800/0x23cab7dad48e5f75 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x75:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x929304787232a39 expref: -99 pid: 70919 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0
00010000:00010000:1.0:1775764822.264999:0:70919:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f1498178800/0x23cab7dad48e5f75 lrc: 2/0,0 mode: PR/PR res: [0x200000405:0x75:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x929304787232a39 expref: -99 pid: 70919 timeout: 0 lvb_type: 3 lru_score: 4 lru_type: 0
00000100:00100000:1.0:1775764822.362491:0:2396:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.110@tcp (source: 12345-192.168.202.110@tcp)
00000100:00100000:1.0:1775764822.369312:0:2396:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.110@tcp (source: 12345-192.168.202.110@tcp)
00000100:00100000:1.0:1775764822.383322:0:39305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.110@tcp, seq: 6393
00000100:00100000:1.0:1775764822.383332:0:39305:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9f14b0fe8850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:LOV_OSC_UUID+5:28873:x1862022124479232:12345-192.168.202.110@tcp:104:
00010000:00010000:1.0:1775764822.383340:0:39305:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast  ns: lustre-OST0000-osc-ffff9f14902b3800 lock: 0000000039155643/0x23cab7dad48e5f9f lrc: 2/0,0 mode: PR/PR res: [0x280000401:0xb6:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800420000000000 nid: local remote: 0x929304787232a7f expref: -99 pid: 70919 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0
00010000:00010000:1.0:1775764822.383370:0:39305:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/3 locks to priority blp list, 3 blwis in pool
00000100:00100000:1.0:1775764822.383375:0:39305:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9f14b0fe8850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:LOV_OSC_UUID+5:28873:x1862022124479232:12345-192.168.202.110@tcp:104: Request processed in 47us (632us total) trans 0 rc 0/0
00000100:00100000:1.0:1775764822.383393:0:39305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.110@tcp, seq: 6393
00010000:00010000:1.0:1775764822.387664:0:60134:0:(ldlm_lockd.c:2794:ldlm_bl_get_work()) Got 1 locks of 0 total in blp.  (0 blwis in pool)
00010000:00010000:1.0:1775764822.387672:0:60134:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-OST0000-osc-ffff9f14902b3800 lock: 00000000cbb11055/0x23cab7dad48e5f21 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0xb6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800420000000000 nid: local remote: 0x92930478723290c expref: -99 pid: 70909 timeout: 0 lvb_type: 1 lru_score: 4 lru_type: 0
00010000:00010000:1.0:1775764822.387682:0:60134:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f149817e000 already unused, calling callback (ffffffffc10b72c0)
00010000:00010000:1.0:1775764822.387688:0:60134:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-OST0000-osc-ffff9f14902b3800 lock: 00000000cbb11055/0x23cab7dad48e5f21 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0xb6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800428400000000 nid: local remote: 0x92930478723290c expref: -99 pid: 70909 timeout: 0 lvb_type: 1 lru_score: 4 lru_type: 0
00000080:00200000:1.0:1775764822.387737:0:60134:0:(vvp_io.c:1817:vvp_io_init()) [0x200000405:0x76:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0
00000080:00200000:1.0:1775764822.387747:0:60134:0:(vvp_io.c:288:vvp_io_fini()) [0x200000405:0x76:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 1
00010000:00010000:1.0:1775764822.387804:0:60134:0:(ldlm_request.c:1438:__ldlm_pack_lock()) ### packing ns: lustre-OST0000-osc-ffff9f14902b3800 lock: 00000000cbb11055/0x23cab7dad48e5f21 lrc: 2/0,0 mode: --/PR res: [0x280000401:0xb6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x804c69400000020 nid: local remote: 0x92930478723290c expref: -99 pid: 70909 timeout: 0 lvb_type: 1 lru_score: 4 lru_type: 0
00010000:00010000:1.0:1775764822.387812:0:60134:0:(ldlm_request.c:1490:ldlm_cancel_pack()) 1 locks packed
00010000:00010000:1.0:1775764822.387837:0:60134:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-OST0000-osc-ffff9f14902b3800 lock: 00000000cbb11055/0x23cab7dad48e5f21 lrc: 1/0,0 mode: --/PR res: [0x280000401:0xb6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x804c69400000020 nid: local remote: 0x92930478723290c expref: -99 pid: 70909 timeout: 0 lvb_type: 1 lru_score: 4 lru_type: 0
00010000:00010000:1.0:1775764822.387845:0:60134:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-OST0000-osc-ffff9f14902b3800 lock: 00000000cbb11055/0x23cab7dad48e5f21 lrc: 0/0,0 mode: --/PR res: [0x280000401:0xb6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x804c69400000020 nid: local remote: 0x92930478723290c expref: -99 pid: 70909 timeout: 0 lvb_type: 1 lru_score: 4 lru_type: 0
00010000:00010000:1.0:1775764822.387861:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00010000:00010000:1.0:1775764822.387863:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00010000:00010000:1.0:1775764822.387865:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775764822.387882:0:2403:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2403:1862022114132608:192.168.202.110@tcp:103:ldlm_bl.0
00000100:00100000:1.0:1775764822.402572:0:39305:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1862022124479488
00000100:00100000:1.0:1775764822.402580:0:39305:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1862022124479488
00000100:00100000:1.0:1775764822.402687:0:39305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.110@tcp, seq: 6394
00000100:00100000:1.0:1775764822.402697:0:39305:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9f14a04d2d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:LOV_OSC_UUID+4:9439:x1862022124479488:12345-192.168.202.110@tcp:105:
00010000:00010000:1.0:1775764822.402711:0:39305:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast  ns: lustre-OST0000-osc-ffff9f14902b3800 lock: 0000000055b0ed3a/0x23cab7dad48e5fb4 lrc: 5/0,1 mode: --/PW res: [0x280000401:0xb6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x20000 nid: local remote: 0x929304787232aa9 expref: -99 pid: 70919 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0
00010000:00010000:1.0:1775764822.402723:0:39305:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-OST0000-osc-ffff9f14902b3800 lock: 0000000055b0ed3a/0x23cab7dad48e5fb4 lrc: 5/0,1 mode: --/PW res: [0x280000401:0xb6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x20000 nid: local remote: 0x929304787232aa9 expref: -99 pid: 70919 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0
00010000:00010000:1.0:1775764822.402732:0:39305:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-OST0000-osc-ffff9f14902b3800 lock: 0000000055b0ed3a/0x23cab7dad48e5fb4 lrc: 5/0,1 mode: --/PW res: [0x280000401:0xb6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10000000020000 nid: local remote: 0x929304787232aa9 expref: -99 pid: 70919 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0
00010000:00010000:1.0:1775764822.402742:0:39305:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-OST0000-osc-ffff9f14902b3800 lock: 0000000055b0ed3a/0x23cab7dad48e5fb4 lrc: 5/0,1 mode: PW/PW res: [0x280000401:0xb6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10080000020000 nid: local remote: 0x929304787232aa9 expref: -99 pid: 70919 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0
00010000:00010000:1.0:1775764822.402751:0:39305:0:(ldlm_resource.c:1714:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-OST0000-osc-ffff9f14902b3800 lock: 0000000055b0ed3a/0x23cab7dad48e5fb4 lrc: 6/0,1 mode: PW/PW res: [0x280000401:0xb6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10080000020000 nid: local remote: 0x929304787232aa9 expref: -99 pid: 70919 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0
00010000:00010000:1.0:1775764822.402761:0:39305:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-OST0000-osc-ffff9f14902b3800 lock: 0000000055b0ed3a/0x23cab7dad48e5fb4 lrc: 6/0,1 mode: PW/PW res: [0x280000401:0xb6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x80000020000 nid: local remote: 0x929304787232aa9 expref: -99 pid: 70919 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0
00010000:00010000:1.0:1775764822.402792:0:39305:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffff9f14c2152600)
00000100:00100000:1.0:1775764822.402857:0:39305:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9f14a04d2d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:LOV_OSC_UUID+4:9439:x1862022124479488:12345-192.168.202.110@tcp:105: Request processed in 161us (1407us total) trans 0 rc 0/0
00000100:00100000:1.0:1775764822.402864:0:39305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.110@tcp, seq: 6394
00000100:00100000:1.0:1775764822.419469:0:2403:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2403:1862022114132608:192.168.202.110@tcp:103:ldlm_bl.0
00000100:00100000:1.0:1775764822.421258:0:39305:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1862022124479744
00000100:00100000:1.0:1775764822.421277:0:39305:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1862022124479744
00000100:00100000:1.0:1775764822.421342:0:39305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.110@tcp, seq: 6396
00000100:00100000:1.0:1775764822.421353:0:39305:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9f14a04d0700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48+5:6487:x1862022124479744:12345-192.168.202.110@tcp:104:
00010000:00010000:1.0:1775764822.421374:0:39305:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast  ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f149817f400/0x23cab7dad48e5fa6 lrc: 2/0,0 mode: CR/CR res: [0x200000405:0x77:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x420000000000 nid: local remote: 0x929304787232a94 expref: -99 pid: 70919 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0
00010000:00010000:1.0:1775764822.421452:0:39305:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool
00000100:00100000:1.0:1775764822.421459:0:39305:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9f14a04d0700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48+5:6487:x1862022124479744:12345-192.168.202.110@tcp:104: Request processed in 108us (367us total) trans 0 rc 0/0
00000100:00100000:1.0:1775764822.421466:0:39305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.110@tcp, seq: 6396
00010000:00010000:1.0:1775764823.306737:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00010000:00010000:1.0:1775764824.334588:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00010000:00010000:1.0:1775764825.365136:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775764826.634894:0:2403:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2403:1862022114135040:192.168.202.110@tcp:400:kworker.0
00000100:00100000:1.0:1775764826.634996:0:2403:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fe680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2403:1862022114135296:192.168.202.110@tcp:400:kworker.0
00000100:00100000:1.0:1775764826.635041:0:2403:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [1->0]
00000100:00100000:1.0:1775764826.635048:0:2403:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2403:1862022114134912:192.168.202.110@tcp:400:kworker.0
00000100:00100000:1.0:1775764826.635075:0:2403:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2403:1862022114135168:192.168.202.110@tcp:400:kworker.0
00000100:00100000:1.0:1775764826.635098:0:2403:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fc700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2403:1862022114135424:192.168.202.110@tcp:400:kworker.0
00010000:00010000:1.0:1775764827.406749:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00010000:00010000:1.0:1775764830.476785:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775764830.738953:0:39305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.110@tcp, seq: 6398
00000100:00100000:1.0:1775764830.738962:0:39305:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9f14a04d0a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41+4:42469:x1862022124486144:12345-192.168.202.110@tcp:104:
00010000:00010000:1.0:1775764830.738969:0:39305:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast  ns: MGC192.168.202.110@tcp lock: ffff9f1498179c00/0x23cab7dad48e5dca lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x1400000000000 nid: local remote: 0x9293047872324a5 expref: -99 pid: 39317 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0
00010000:00010000:1.0:1775764830.739013:0:39305:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to priority blp list, 2 blwis in pool
00000100:00100000:1.0:1775764830.739018:0:39305:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9f14a04d0a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41+3:42469:x1862022124486144:12345-192.168.202.110@tcp:104: Request processed in 59us (260us total) trans 0 rc 0/0
00000100:00100000:1.0:1775764830.739024:0:39305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.110@tcp, seq: 6398
00010000:00010000:1.0:1775764830.739043:0:60132:0:(ldlm_lockd.c:2794:ldlm_bl_get_work()) Got 1 locks of 1 total in blp.  (1 blwis in pool)
00010000:00010000:1.0:1775764830.739048:0:60132:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.202.110@tcp lock: ffff9f1498179200/0x23cab7dad48e5dd1 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x1400000000000 nid: local remote: 0x9293047872324ac expref: -99 pid: 39317 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0
00010000:00010000:1.0:1775764830.739055:0:60132:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f1498179200 already unused, calling callback (ffffffffc123ac20)
10000000:00010000:1.0:1775764830.739059:0:60132:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.202.110@tcp lock: ffff9f1498179200/0x23cab7dad48e5dd1 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x1400400000000 nid: local remote: 0x9293047872324ac expref: -99 pid: 39317 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0
00010000:00010000:1.0:1775764830.739068:0:60132:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.202.110@tcp lock: ffff9f1498179200/0x23cab7dad48e5dd1 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x1408400000000 nid: local remote: 0x9293047872324ac expref: -99 pid: 39317 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0
10000000:00010000:1.0:1775764830.739075:0:60132:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.202.110@tcp lock: ffff9f1498179200/0x23cab7dad48e5dd1 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x1409400000000 nid: local remote: 0x9293047872324ac expref: -99 pid: 39317 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0
10000000:01000000:1.0:1775764830.739080:0:60132:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre)
00010000:00010000:1.0:1775764830.739112:0:60132:0:(ldlm_request.c:1438:__ldlm_pack_lock()) ### packing ns: MGC192.168.202.110@tcp lock: ffff9f1498179200/0x23cab7dad48e5dd1 lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x5c09400000020 nid: local remote: 0x9293047872324ac expref: -99 pid: 39317 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0
00010000:00010000:1.0:1775764830.739118:0:60132:0:(ldlm_request.c:1490:ldlm_cancel_pack()) 1 locks packed
00010000:00010000:1.0:1775764830.739138:0:60132:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.202.110@tcp lock: ffff9f1498179200/0x23cab7dad48e5dd1 lrc: 1/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x5c09400000020 nid: local remote: 0x9293047872324ac expref: -99 pid: 39317 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0
00010000:00010000:1.0:1775764830.739145:0:60132:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.202.110@tcp lock: ffff9f1498179200/0x23cab7dad48e5dd1 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x5c09400000020 nid: local remote: 0x9293047872324ac expref: -99 pid: 39317 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0
00010000:00010000:1.0:1775764830.739163:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00010000:00010000:1.0:1775764830.739164:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00010000:00010000:1.0:1775764830.739166:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775764830.739180:0:2404:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2404:1862022114135680:192.168.202.110@tcp:103:ldlm_bl.0
00000100:00100000:1.0:1775764830.749727:0:2404:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2404:1862022114135680:192.168.202.110@tcp:103:ldlm_bl.0
10000000:01000000:1.0:1775764831.882686:0:39317:0:(mgc_request.c:611:do_requeue()) updating log lustre-client
10000000:01000000:1.0:1775764831.882693:0:39317:0:(mgc_request.c:1819:mgc_process_log()) Process log lustre-client-ffff9f14902b3800 from 70
10000000:01000000:1.0:1775764831.882699:0:39317:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-client (res 0x65727473756c)
00010000:00010000:1.0:1775764831.883081:0:39317:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.202.110@tcp lock: ffff9f14ab4c4c00/0x23cab7dad48e5fd7 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 39317 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0
00010000:00010000:1.0:1775764831.883094:0:39317:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.202.110@tcp lock: ffff9f14ab4c4c00/0x23cab7dad48e5fd7 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 39317 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0
00010000:00010000:1.0:1775764831.883107:0:39317:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.202.110@tcp lock: ffff9f14ab4c4c00/0x23cab7dad48e5fd7 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 39317 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0
00000100:00100000:1.0:1775764831.883140:0:39317:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:39317:1862022114136448:192.168.202.110@tcp:101:ll_cfg_requeue.0
00000100:00100000:1.0:1775764831.883255:0:39317:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f149180db40 going to sleep for 11 seconds
00000020:01000000:1.0:1775764831.926201:0:39317:0:(obd_config.c:2143:class_config_parse_llog()) Processed log lustre-client gen 70-72 (rc=0)
10000000:01000000:1.0:1775764831.926218:0:39317:0:(mgc_request.c:1896:mgc_process_log()) MGC192.168.202.110@tcp: configuration from log 'lustre-client' succeeded (0).
00010000:00010000:1.0:1775764831.926229:0:39317:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.202.110@tcp lock: ffff9f14ab4c4c00/0x23cab7dad48e5fd7 lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x929304787232b35 expref: -99 pid: 39317 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0
00010000:00010000:1.0:1775764831.926239:0:39317:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.202.110@tcp lock: ffff9f14ab4c4c00/0x23cab7dad48e5fd7 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x929304787232b35 expref: -99 pid: 39317 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0
10000000:01000000:1.0:1775764831.926252:0:39317:0:(mgc_request.c:611:do_requeue()) updating log lustre-sptlrpc
10000000:01000000:1.0:1775764831.926254:0:39317:0:(mgc_request.c:1819:mgc_process_log()) Process log lustre-sptlrpc-ffff9f149022cd40 from 1
10000000:01000000:1.0:1775764831.926256:0:39317:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-sptlrpc (res 0x65727473756c)
00010000:00010000:1.0:1775764831.926288:0:39317:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.202.110@tcp lock: ffff9f14ab4c4000/0x23cab7dad48e5fde lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 39317 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0
00010000:00010000:1.0:1775764831.926295:0:39317:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.202.110@tcp lock: ffff9f14ab4c4000/0x23cab7dad48e5fde lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 39317 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0
00010000:00010000:1.0:1775764831.926305:0:39317:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.202.110@tcp lock: ffff9f14ab4c4000/0x23cab7dad48e5fde lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 39317 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0
00000100:00100000:1.0:1775764831.926329:0:39317:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdf80 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:39317:1862022114137088:192.168.202.110@tcp:101:ll_cfg_requeue.0
00000100:00100000:1.0:1775764831.926418:0:39317:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f149180df00 going to sleep for 11 seconds
00010000:00010000:1.0:1775764832.526179:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00010000:00010000:1.0:1775764835.594862:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775764835.837068:0:39304:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.110@tcp, seq: 6400
00000100:00100000:1.0:1775764835.837089:0:39304:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9f149959e050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41+3:42575:x1862022124495616:12345-192.168.202.110@tcp:104:
00010000:00010000:1.0:1775764835.837103:0:39304:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast  ns: MGC192.168.202.110@tcp lock: ffff9f14ab4c4c00/0x23cab7dad48e5fd7 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1400000000000 nid: local remote: 0x929304787232b35 expref: -99 pid: 39317 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0
00010000:00010000:1.0:1775764835.837187:0:39304:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool
00000100:00100000:1.0:1775764835.837194:0:39304:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9f149959e050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41+3:42575:x1862022124495616:12345-192.168.202.110@tcp:104: Request processed in 115us (1888us total) trans 0 rc 0/0
00000100:00100000:1.0:1775764835.837202:0:39304:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.110@tcp, seq: 6400
00000100:00100000:1.0:1775764836.881019:0:2403:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2403:1862022114137600:192.168.202.110@tcp:400:kworker.0
00000100:00100000:1.0:1775764836.899086:0:2403:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2403:1862022114138112:192.168.202.110@tcp:400:kworker.0
00000100:00100000:1.0:1775764836.899107:0:2403:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2403:1862022114137984:192.168.202.110@tcp:400:kworker.0
00000100:00100000:1.0:1775764836.899118:0:2403:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2403:1862022114137856:192.168.202.110@tcp:400:kworker.0
00010000:00010000:1.0:1775764837.643287:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000080:00200000:1.0:1775764839.342382:0:71578:0:(file.c:6121:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9f1491deba88),name=/
00000002:00010000:1.0:1775764839.342397:0:71578:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0
00010000:00010000:1.0:1775764839.342421:0:71578:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f149153fe00/0x23cab7dad48e5fc2 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x929304787232ad3 expref: -99 pid: 70921 timeout: 0 lvb_type: 0 lru_score: 9 lru_type: 1
00010000:00010000:1.0:1775764839.342433:0:71578:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f149153fe00/0x23cab7dad48e5fc2 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x929304787232ad3 expref: -99 pid: 70921 timeout: 0 lvb_type: 0 lru_score: 9 lru_type: 1
00000080:00200000:1.0:1775764839.342439:0:71578:0:(namei.c:953:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1
00000080:00010000:1.0:1775764839.342444:0:71578:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9f1491deba88)
00000080:00010000:1.0:1775764839.342447:0:71578:0:(llite_internal.h:1984:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9f1491deba88) for lock 0x23cab7dad48e5fc2
00000080:00010000:1.0:1775764839.342452:0:71578:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x23cab7dad48e5fc2 from it ffffb06c01e2fbf8
00010000:00010000:1.0:1775764839.342455:0:71578:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f149153fe00/0x23cab7dad48e5fc2 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x929304787232ad3 expref: -99 pid: 70921 timeout: 0 lvb_type: 0 lru_score: 9 lru_type: 1
00010000:00010000:1.0:1775764839.342463:0:71578:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f149153fe00/0x23cab7dad48e5fc2 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x929304787232ad3 expref: -99 pid: 70921 timeout: 0 lvb_type: 0 lru_score: 10 lru_type: 1
00000080:00200000:1.0:1775764839.342474:0:71578:0:(file.c:6593:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9f1491deba88), inode mode 41ed mask 1
00000080:00200000:1.0:1775764839.342481:0:71578:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=f206.sanity-flr, flags=777
00000080:00200000:1.0:1775764839.342485:0:71578:0:(file.c:6121:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9f1491deba88),name=/
00000002:00010000:1.0:1775764839.342491:0:71578:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0
00010000:00010000:1.0:1775764839.342497:0:71578:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f149153fe00/0x23cab7dad48e5fc2 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x929304787232ad3 expref: -99 pid: 70921 timeout: 0 lvb_type: 0 lru_score: 10 lru_type: 1
00010000:00010000:1.0:1775764839.342505:0:71578:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f149153fe00/0x23cab7dad48e5fc2 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x929304787232ad3 expref: -99 pid: 70921 timeout: 0 lvb_type: 0 lru_score: 10 lru_type: 1
00000080:00200000:1.0:1775764839.342512:0:71578:0:(namei.c:953:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1
00000080:00010000:1.0:1775764839.342515:0:71578:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9f1491deba88)
00000080:00010000:1.0:1775764839.342517:0:71578:0:(llite_internal.h:1984:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9f1491deba88) for lock 0x23cab7dad48e5fc2
00000080:00010000:1.0:1775764839.342520:0:71578:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x23cab7dad48e5fc2 from it ffffb06c01e2fb38
00010000:00010000:1.0:1775764839.342523:0:71578:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f149153fe00/0x23cab7dad48e5fc2 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x929304787232ad3 expref: -99 pid: 70921 timeout: 0 lvb_type: 0 lru_score: 10 lru_type: 1
00010000:00010000:1.0:1775764839.342530:0:71578:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f149153fe00/0x23cab7dad48e5fc2 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x929304787232ad3 expref: -99 pid: 70921 timeout: 0 lvb_type: 0 lru_score: 11 lru_type: 1
00000080:00200000:1.0:1775764839.342538:0:71578:0:(file.c:6593:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9f1491deba88), inode mode 41ed mask 3
00000080:00200000:1.0:1775764839.342543:0:71578:0:(namei.c:1409:ll_atomic_open()) VFS Op:name=f206.sanity-flr, dir=[0x200000007:0x1:0x0](ffff9f1491deba88), file ffff9f1491539200, open_flags a141, mode 81b6 opened 0
00000080:00200000:1.0:1775764839.342793:0:71578:0:(namei.c:1073:ll_lookup_it()) VFS Op:name=f206.sanity-flr, dir=[0x200000007:0x1:0x0](ffff9f1491deba88), intent=open|creat
00000002:00010000:1.0:1775764839.342816:0:71578:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: f206.sanity-flr,[0x200000405:0x78:0x0]) in obj [0x200000007:0x1:0x0], intent: open|creat flags 0120502
00010000:00010000:1.0:1775764839.342829:0:71578:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f149153fe00/0x23cab7dad48e5fc2 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x929304787232ad3 expref: -99 pid: 70921 timeout: 0 lvb_type: 0 lru_score: 11 lru_type: 1
00010000:00010000:1.0:1775764839.342882:0:71578:0:(ldlm_request.c:1438:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f149153fe00/0x23cab7dad48e5fc2 lrc: 1/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x929304787232ad3 expref: -99 pid: 70921 timeout: 0 lvb_type: 0 lru_score: 11 lru_type: 1
00010000:00010000:1.0:1775764839.342889:0:71578:0:(ldlm_request.c:1490:ldlm_cancel_pack()) 1 locks packed
00010000:00010000:1.0:1775764839.342892:0:71578:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f149153fe00/0x23cab7dad48e5fc2 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x929304787232ad3 expref: -99 pid: 70921 timeout: 0 lvb_type: 0 lru_score: 11 lru_type: 1
00010000:00010000:1.0:1775764839.342933:0:71578:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f14ab4c5600/0x23cab7dad48e5ffa lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 71578 initiator: MDT0 lru_score: 0 lru_type: 0
00010000:00010000:1.0:1775764839.342940:0:71578:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f14ab4c5600/0x23cab7dad48e5ffa lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 71578 initiator: MDT0 lru_score: 0 lru_type: 0
00000020:00100000:1.0:1775764839.342950:0:71578:0:(genops.c:2442:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9f14902b3800: modify RPC slot 1 is allocated opc 101, max 7
00000100:00100000:1.0:1775764839.342962:0:71578:0:(client.c:806:ptlrpc_reassign_next_xid()) @@@ reassign xid  req@ffff9f14ab4fdc00 x1862022114139392/t0(0) o101->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 528/67824 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0
00010000:00010000:1.0:1775764839.342971:0:71578:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f14ab4c5600/0x23cab7dad48e5ffa lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 71578 initiator: MDT0 lru_score: 0 lru_type: 0
00000100:00100000:1.0:1775764839.342994:0:71578:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job lfs:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:71578:1862022114139392:192.168.202.110@tcp:101:lfs.0
00000100:00100000:1.0:1775764839.343178:0:71578:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f149180d540 going to sleep for 50 seconds
00000100:00100000:1.0:1775764839.357215:0:2395:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.110@tcp (source: 12345-192.168.202.110@tcp)
00000080:00200000:1.0:1775764839.436783:0:71579:0:(file.c:6121:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9f1491deba88),name=/
00000002:00010000:1.0:1775764839.436832:0:71579:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0
00010000:00010000:1.0:1775764839.436912:0:71579:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f14ab4c5000/0x23cab7dad48e600f lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 71579 initiator: MDT0 lru_score: 0 lru_type: 0
00010000:00010000:1.0:1775764839.436923:0:71579:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f14ab4c5000/0x23cab7dad48e600f lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 71579 initiator: MDT0 lru_score: 0 lru_type: 0
00010000:00010000:1.0:1775764839.436934:0:71579:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f14ab4c5000/0x23cab7dad48e600f lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 71579 initiator: MDT0 lru_score: 0 lru_type: 0
00000100:00100000:1.0:1775764839.436964:0:71579:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fe300 pname:cluuid:pid:xid:nid:opc:job lfs:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:71579:1862022114140544:192.168.202.110@tcp:101:lfs.0
00000100:00100000:1.0:1775764839.437081:0:71579:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f149180d9c0 going to sleep for 50 seconds
00000100:00100000:1.0:1775764839.456802:0:2404:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2404:1862022114140928:192.168.202.110@tcp:101:lfs.0
00000080:00200000:1.0:1775764839.466441:0:71579:0:(file.c:1742:ll_merge_attr_nolock()) [0x200000405:0x78:0x0] updating i_size 0 i_blocks 0
00000020:00010000:1.0:1775764839.466450:0:71579:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff9f14983b3230 (ffff9f14861432d8/0) at cl_lock_release():245
00000020:00010000:1.0:1775764839.466453:0:71579:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9f14983b3230 (ffff9f14861432d8/0) at cl_lock_cancel():142
00000020:00010000:1.0:1775764839.466458:0:71579:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9f14981b2830 (ffff9f1486143008/1) at cl_lock_cancel():142
00000020:00010000:1.0:1775764839.466461:0:71579:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9f14983b3230 (ffff9f14861432d8/0) at cl_lock_fini():72
00000020:00010000:1.0:1775764839.466464:0:71579:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9f14981b2830 (ffff9f14861432d8/1) at cl_lock_fini():72
00000080:00200000:1.0:1775764839.466501:0:71579:0:(vvp_io.c:288:vvp_io_fini()) [0x200000405:0x78:0x0] ignore/verify layout 0/1, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0
00000080:00200008:1.0:1775764839.466701:0:71579:0:(file.c:447:ll_file_release()) START file f206.sanity-flr:[0x200000405:0x78:0x0](ffff9f14a10b7448), flags 154000
00000002:00100000:1.0:1775764839.466770:0:71579:0:(mdc_request.c:964:mdc_close()) @@@ matched open  req@ffff9f1491686680 x1862022114140800/t8589936628(8589936628) o101->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 600/680 e 0 to 0 dl 1775764894 ref 1 fl Complete:RPQU/604/ffffffff rc 0/-1 job:'lfs.0' uid:0 gid:0 projid:0
00000020:00100000:1.0:1775764839.466782:0:71579:0:(genops.c:2442:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9f14902b3800: modify RPC slot 1 is allocated opc 35, max 7
00000100:00100000:1.0:1775764839.466786:0:71579:0:(client.c:806:ptlrpc_reassign_next_xid()) @@@ reassign xid  req@ffff9f14ab4fea00 x1862022114141184/t0(0) o35->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:23/10 lens 392/2128 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0
00000100:00100000:1.0:1775764839.466811:0:71579:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job lfs:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:71579:1862022114141184:192.168.202.110@tcp:35:lfs.0
00000100:00100000:1.0:1775764839.466890:0:71579:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f149180da80 going to sleep for 11 seconds
00000100:00100000:1.0:1775764839.473041:0:2396:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.110@tcp (source: 12345-192.168.202.110@tcp)
00000100:00100000:1.0:1775764839.502484:0:2395:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.110@tcp (source: 12345-192.168.202.110@tcp)
00000100:00100000:1.0:1775764839.553511:0:2395:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.110@tcp (source: 12345-192.168.202.110@tcp)
00000080:00200008:1.0:1775764839.715964:0:71579:0:(file.c:5170:ll_file_seek()) START file f206.sanity-flr:[0x200000405:0x78:0x0], offset: 0, type: (null)
00000080:00200000:1.0:1775764839.716012:0:71579:0:(vvp_io.c:1817:vvp_io_init()) [0x200000405:0x78:0x0] ignore/verify layout 0/1, layout version 0 restore needed 0
00000001:00010000:1.0:1775764839.716022:0:71579:0:(glimpse.c:73:cl_glimpse_lock()) Glimpsing inode [0x200000405:0x78:0x0]
00000008:00010000:1.0:1775764839.716049:0:71579:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: a
00000008:00010000:1.0:1775764839.716053:0:71579:0:(osc_lock.c:1310:osc_lock_init()) ### lock ffff9f14945e0cb0, osc lock ffff9f148759ad10, flags 0x400001002
00010000:00010000:1.0:1775764839.716081:0:71579:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-OST0001-osc-ffff9f14902b3800 lock: 00000000255d8ffe/0x23cab7dad48e601d lrc: 2/1,0 mode: PR/PR res: [0x2c0000401:0xb7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800020000000000 nid: local remote: 0x929304787232ce0 expref: -99 pid: 71579 timeout: 0 lvb_type: 1 lru_score: 5 lru_type: 0
00010000:00010000:1.0:1775764839.716110:0:71579:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 18446744073709551615) ns: lustre-OST0001-osc-ffff9f14902b3800 lock: 00000000255d8ffe/0x23cab7dad48e601d lrc: 2/1,0 mode: PR/PR res: [0x2c0000401:0xb7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800020000000000 nid: local remote: 0x929304787232ce0 expref: -99 pid: 71579 timeout: 0 lvb_type: 1 lru_score: 5 lru_type: 0
00000020:00200000:1.0:1775764839.716126:0:71579:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining.
00010000:00010000:1.0:1775764839.716131:0:71579:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-OST0001-osc-ffff9f14902b3800 lock: 00000000255d8ffe/0x23cab7dad48e601d lrc: 5/1,0 mode: PR/PR res: [0x2c0000401:0xb7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x810020000000000 nid: local remote: 0x929304787232ce0 expref: -99 pid: 71579 timeout: 0 lvb_type: 1 lru_score: 5 lru_type: 0
00010000:00010000:1.0:1775764839.716142:0:71579:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-OST0001-osc-ffff9f14902b3800 lock: 00000000255d8ffe/0x23cab7dad48e601d lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0xb7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800020000000000 nid: local remote: 0x929304787232ce0 expref: -99 pid: 71579 timeout: 0 lvb_type: 1 lru_score: 6 lru_type: 0
00000020:00200000:1.0:1775764839.716158:0:71579:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining.
00000080:00200000:1.0:1775764839.716169:0:71579:0:(file.c:1742:ll_merge_attr_nolock()) [0x200000405:0x78:0x0] updating i_size 0 i_blocks 0
00000020:00010000:1.0:1775764839.716173:0:71579:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff9f1486116b58 (ffff9f1487c50cf8/0) at cl_lock_release():245
00000020:00010000:1.0:1775764839.716176:0:71579:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9f1486116b58 (ffff9f1487c50cf8/0) at cl_lock_cancel():142
00000020:00010000:1.0:1775764839.716179:0:71579:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9f14945e0cb0 (ffff9f1487c501b8/1) at cl_lock_cancel():142
00000020:00010000:1.0:1775764839.716184:0:71579:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9f1486116b58 (ffff9f1487c50cf8/0) at cl_lock_fini():72
00000020:00010000:1.0:1775764839.716186:0:71579:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9f14945e0cb0 (ffff9f1487c50cf8/1) at cl_lock_fini():72
00000080:00200000:1.0:1775764839.716216:0:71579:0:(vvp_io.c:288:vvp_io_fini()) [0x200000405:0x78:0x0] ignore/verify layout 0/1, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0
00000080:00200008:1.0:1775764839.716231:0:71579:0:(file.c:5201:ll_file_seek()) COMPLETED file f206.sanity-flr:[0x200000405:0x78:0x0], offset: 0, type: (null), rc = 0
00000080:00200008:1.0:1775764839.716235:0:71579:0:(file.c:5170:ll_file_seek()) START file f206.sanity-flr:[0x200000405:0x78:0x0], offset: 0, type: SEEK_HOLE
00000080:00200000:1.0:1775764839.716247:0:71579:0:(vvp_io.c:1817:vvp_io_init()) [0x200000405:0x78:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0
00020000:00200000:1.0:1775764839.716253:0:71579:0:(lov_io.c:860:lov_io_iter_init()) component[0] flags 0x10
00000080:00200000:1.0:1775764839.716273:0:71579:0:(file.c:1742:ll_merge_attr_nolock()) [0x200000405:0x78:0x0] updating i_size 0 i_blocks 0
00000080:00200000:1.0:1775764839.716285:0:71579:0:(vvp_io.c:288:vvp_io_fini()) [0x200000405:0x78:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0
00000080:00200000:1.0:1775764839.716301:0:71579:0:(file.c:1815:ll_io_set_mirror()) f206.sanity-flr: desiginated mirror: 0
00000080:00200000:1.0:1775764839.716304:0:71579:0:(vvp_io.c:1817:vvp_io_init()) [0x200000405:0x78:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0
00020000:00200000:1.0:1775764839.716308:0:71579:0:(lov_io.c:860:lov_io_iter_init()) component[0] flags 0x10
00000080:00200000:1.0:1775764839.718429:0:71579:0:(vvp_io.c:198:vvp_io_one_lock_index()) lock: 0 [0, 18446744073709551615]
00000008:00010000:1.0:1775764839.718466:0:71579:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 8
00000008:00010000:1.0:1775764839.718470:0:71579:0:(osc_lock.c:1310:osc_lock_init()) ### lock ffff9f14945e0cb0, osc lock ffff9f148759ad10, flags 0x0
00010000:00010000:1.0:1775764839.718498:0:71579:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-OST0001-osc-ffff9f14902b3800 lock: 00000000255d8ffe/0x23cab7dad48e601d lrc: 2/1,0 mode: PR/PR res: [0x2c0000401:0xb7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800020000000000 nid: local remote: 0x929304787232ce0 expref: -99 pid: 71579 timeout: 0 lvb_type: 1 lru_score: 6 lru_type: 0
00010000:00010000:1.0:1775764839.718514:0:71579:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 18446744073709551615) ns: lustre-OST0001-osc-ffff9f14902b3800 lock: 00000000255d8ffe/0x23cab7dad48e601d lrc: 2/1,0 mode: PR/PR res: [0x2c0000401:0xb7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800020000000000 nid: local remote: 0x929304787232ce0 expref: -99 pid: 71579 timeout: 0 lvb_type: 1 lru_score: 6 lru_type: 0
00010000:00010000:1.0:1775764839.718527:0:71579:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-OST0001-osc-ffff9f14902b3800 lock: 00000000255d8ffe/0x23cab7dad48e601d lrc: 6/2,0 mode: PR/PR res: [0x2c0000401:0xb7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x810020000000000 nid: local remote: 0x929304787232ce0 expref: -99 pid: 71579 timeout: 0 lvb_type: 1 lru_score: 6 lru_type: 0
00010000:00010000:1.0:1775764839.718536:0:71579:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-OST0001-osc-ffff9f14902b3800 lock: 00000000255d8ffe/0x23cab7dad48e601d lrc: 6/2,0 mode: PR/PR res: [0x2c0000401:0xb7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x810020000000000 nid: local remote: 0x929304787232ce0 expref: -99 pid: 71579 timeout: 0 lvb_type: 1 lru_score: 6 lru_type: 0
00010000:00010000:1.0:1775764839.718544:0:71579:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-OST0001-osc-ffff9f14902b3800 lock: 00000000255d8ffe/0x23cab7dad48e601d lrc: 5/1,0 mode: PR/PR res: [0x2c0000401:0xb7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x810020000000000 nid: local remote: 0x929304787232ce0 expref: -99 pid: 71579 timeout: 0 lvb_type: 1 lru_score: 6 lru_type: 0
00000080:00200000:1.0:1775764839.718725:0:71579:0:(file.c:1742:ll_merge_attr_nolock()) [0x200000405:0x78:0x0] updating i_size 0 i_blocks 0
00000020:00010000:1.0:1775764839.718733:0:71579:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff9f14a67667b8 (ffff9f1487c50cf8/0) at cl_lock_release():245
00000020:00010000:1.0:1775764839.718737:0:71579:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9f14a67667b8 (ffff9f1487c50cf8/0) at cl_lock_cancel():142
00000020:00010000:1.0:1775764839.718740:0:71579:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9f14945e0cb0 (ffff9f1487c501b8/1) at cl_lock_cancel():142
00010000:00010000:1.0:1775764839.718744:0:71579:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-OST0001-osc-ffff9f14902b3800 lock: 00000000255d8ffe/0x23cab7dad48e601d lrc: 4/1,0 mode: PR/PR res: [0x2c0000401:0xb7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x810020000000000 nid: local remote: 0x929304787232ce0 expref: -99 pid: 71579 timeout: 0 lvb_type: 1 lru_score: 6 lru_type: 0
00010000:00010000:1.0:1775764839.718753:0:71579:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-OST0001-osc-ffff9f14902b3800 lock: 00000000255d8ffe/0x23cab7dad48e601d lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0xb7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800020000000000 nid: local remote: 0x929304787232ce0 expref: -99 pid: 71579 timeout: 0 lvb_type: 1 lru_score: 7 lru_type: 1
00000020:00010000:1.0:1775764839.718769:0:71579:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9f14a67667b8 (ffff9f1487c50cf8/0) at cl_lock_fini():72
00000020:00010000:1.0:1775764839.718771:0:71579:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9f14945e0cb0 (ffff9f1487c50cf8/1) at cl_lock_fini():72
00000080:00200000:1.0:1775764839.718807:0:71579:0:(vvp_io.c:288:vvp_io_fini()) [0x200000405:0x78:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0
00000080:00200080:1.0:1775764839.718840:0:71579:0:(file.c:4602:ll_file_ioctl()) VFS Op:inode=[0x200000405:0x78:0x0](ffff9f14a10b7448) cmd=66f4 arg=8ae
00000080:00200008:1.0:1775764839.718859:0:71579:0:(file.c:2454:do_file_read_iter()) START file f206.sanity-flr:[0x200000405:0x78:0x0], ppos: 0, count: 67108864
00000080:00200000:1.0:1775764839.718885:0:71579:0:(file.c:2058:ll_file_io_generic()) f206.sanity-flr: read ppos: 0, bytes: 67108864
00000080:00200000:1.0:1775764839.720276:0:71579:0:(file.c:1815:ll_io_set_mirror()) f206.sanity-flr: desiginated mirror: 0
00000020:00200000:1.0:1775764839.720292:0:71579:0:(cl_io.c:214:cl_io_rw_init()) header@00000000df2a508b[0x110, 2, [0x200000405:0x78:0x0] hash]
00000020:00200000:1.0:1775764839.720295:0:71579:0:(cl_io.c:214:cl_io_rw_init()) io range: 1 [0, 67108864) 2048 0
00000080:00200000:1.0:1775764839.720301:0:71579:0:(vvp_io.c:1817:vvp_io_init()) [0x200000405:0x78:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0
00000080:00200000:1.0:1775764839.720468:0:71579:0:(file.c:2136:ll_file_io_generic()) Range lock [0, 16383]
00020000:00200000:1.0:1775764839.720479:0:71579:0:(lov_io.c:1052:lov_io_rw_iter_init()) stripe: 0 chunk: [0, 67108864) 67108864, 67108864
00020000:00200000:1.0:1775764839.720484:0:71579:0:(lov_io.c:860:lov_io_iter_init()) component[0] flags 0x10
00000080:00200000:1.0:1775764839.720533:0:71579:0:(vvp_io.c:198:vvp_io_one_lock_index()) lock: 0 [0, 16383]
00000008:00010000:1.0:1775764839.720545:0:71579:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 11
00000008:00010000:1.0:1775764839.720549:0:71579:0:(osc_lock.c:1310:osc_lock_init()) ### lock ffff9f14945e0cb0, osc lock ffff9f148759ad10, flags 0x40000
00000080:00200000:1.0:1775764839.720583:0:71579:0:(vvp_io.c:826:vvp_io_read_start()) f206.sanity-flr: read [0, 67108864)
00000001:00010000:1.0:1775764839.720592:0:71579:0:(glimpse.c:73:cl_glimpse_lock()) Glimpsing inode [0x200000405:0x78:0x0]
00000008:00010000:1.0:1775764839.720598:0:71579:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: a
00000008:00010000:1.0:1775764839.720600:0:71579:0:(osc_lock.c:1310:osc_lock_init()) ### lock ffff9f14945e00b0, osc lock ffff9f148759a000, flags 0x400001002
00010000:00010000:1.0:1775764839.720618:0:71579:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-OST0001-osc-ffff9f14902b3800 lock: 00000000255d8ffe/0x23cab7dad48e601d lrc: 2/1,0 mode: PR/PR res: [0x2c0000401:0xb7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800020000000000 nid: local remote: 0x929304787232ce0 expref: -99 pid: 71579 timeout: 0 lvb_type: 1 lru_score: 7 lru_type: 1
00010000:00010000:1.0:1775764839.720633:0:71579:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 18446744073709551615) ns: lustre-OST0001-osc-ffff9f14902b3800 lock: 00000000255d8ffe/0x23cab7dad48e601d lrc: 2/1,0 mode: PR/PR res: [0x2c0000401:0xb7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800020000000000 nid: local remote: 0x929304787232ce0 expref: -99 pid: 71579 timeout: 0 lvb_type: 1 lru_score: 7 lru_type: 1
00000020:00200000:1.0:1775764839.720645:0:71579:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining.
00010000:00010000:1.0:1775764839.720648:0:71579:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-OST0001-osc-ffff9f14902b3800 lock: 00000000255d8ffe/0x23cab7dad48e601d lrc: 5/1,0 mode: PR/PR res: [0x2c0000401:0xb7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x810020000000000 nid: local remote: 0x929304787232ce0 expref: -99 pid: 71579 timeout: 0 lvb_type: 1 lru_score: 7 lru_type: 1
00010000:00010000:1.0:1775764839.720657:0:71579:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-OST0001-osc-ffff9f14902b3800 lock: 00000000255d8ffe/0x23cab7dad48e601d lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0xb7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800020000000000 nid: local remote: 0x929304787232ce0 expref: -99 pid: 71579 timeout: 0 lvb_type: 1 lru_score: 8 lru_type: 1
00000020:00200000:1.0:1775764839.720669:0:71579:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining.
00000080:00200000:1.0:1775764839.720675:0:71579:0:(file.c:1742:ll_merge_attr_nolock()) [0x200000405:0x78:0x0] updating i_size 0 i_blocks 0
00000020:00010000:1.0:1775764839.720679:0:71579:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff9f1486116b58 (ffff9f1487c50cf8/0) at cl_lock_release():245
00000020:00010000:1.0:1775764839.720682:0:71579:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9f1486116b58 (ffff9f1487c50cf8/0) at cl_lock_cancel():142
00000020:00010000:1.0:1775764839.720685:0:71579:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9f14945e00b0 (ffff9f1487c501b8/1) at cl_lock_cancel():142
00000020:00010000:1.0:1775764839.720689:0:71579:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9f1486116b58 (ffff9f1487c50cf8/0) at cl_lock_fini():72
00000020:00010000:1.0:1775764839.720691:0:71579:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9f14945e00b0 (ffff9f1487c50cf8/1) at cl_lock_fini():72
00000080:00200000:1.0:1775764839.720698:0:71579:0:(vvp_io.c:885:vvp_io_read_start()) count:67108864 iocb pos:0
00000080:00200000:1.0:1775764839.720704:0:71579:0:(rw26.c:395:ll_direct_IO()) VFS Op:inode=[0x200000405:0x78:0x0](ffff9f14a10b7448), size=67108864 (max 536870912), offset=0=0x0, pages 16384 (max 131072), lockless, parallel
00000020:00010000:1.0:1775764839.720713:0:71579:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff9f14a67667b8 (ffff9f1487c50cf8/0) at cl_lock_release():245
00000020:00010000:1.0:1775764839.720715:0:71579:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9f14a67667b8 (ffff9f1487c50cf8/0) at cl_lock_cancel():142
00000020:00010000:1.0:1775764839.720718:0:71579:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9f14945e0cb0 (ffff9f1487c501b8/1) at cl_lock_cancel():142
00000020:00010000:1.0:1775764839.720720:0:71579:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9f14a67667b8 (ffff9f1487c50cf8/0) at cl_lock_fini():72
00000020:00010000:1.0:1775764839.720723:0:71579:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9f14945e0cb0 (ffff9f1487c50cf8/1) at cl_lock_fini():72
00000080:00200000:1.0:1775764839.720727:0:71579:0:(vvp_io.c:517:vvp_io_advance()) advancing 0 bytes
00000020:00200000:1.0:1775764839.720732:0:71579:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining.
00000080:00200000:1.0:1775764839.720735:0:71579:0:(file.c:2173:ll_file_io_generic()) Range unlock [0, 16383]
00000080:00200000:1.0:1775764839.720756:0:71579:0:(vvp_io.c:288:vvp_io_fini()) [0x200000405:0x78:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0
00000080:00200000:1.0:1775764839.720760:0:71579:0:(file.c:2206:ll_file_io_generic()) f206.sanity-flr: 1 io complete with rc: 0, result: 0, restart: 0
00000020:00200000:1.0:1775764839.720762:0:71579:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining.
00000080:00200000:1.0:1775764839.720767:0:71579:0:(file.c:2292:ll_file_io_generic()) iot: 1, result: 0
00000080:00200000:1.0:1775764839.720792:0:71579:0:(file.c:5304:ll_fsync()) VFS Op:inode=[0x200000405:0x79:0x0](ffff9f14a11100c8), start 0, end 9223372036854775807, datasync 0
00000100:00100000:1.0:1775764839.720848:0:71579:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job lfs:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:71579:1862022114143488:192.168.202.110@tcp:44:lfs.0
00000100:00100000:1.0:1775764839.720960:0:71579:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f149180d0c0 going to sleep for 50 seconds
00000080:00200000:1.0:1775764839.748170:0:71579:0:(file.c:1742:ll_merge_attr_nolock()) [0x200000405:0x79:0x0] updating i_size 0 i_blocks 0
00000080:00200000:1.0:1775764839.748210:0:71579:0:(vvp_io.c:288:vvp_io_fini()) [0x200000405:0x79:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0
00000100:00100000:1.0:1775764839.768995:0:2404:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2404:1862022114143744:192.168.202.110@tcp:1:lfs.0
00000080:00200008:1.0:1775764839.776983:0:71579:0:(llite_lib.c:2274:ll_setattr_raw()) START file .:VOLATILE:0000:5037705A:fd=03:[0x200000405:0x79:0x0](ffff9f14a11100c8) current size 0, valid attrs 101f0, mode 9f14, uid 0, gid 0, new size 4294967296, atime 1775764839.000000000, mtime 1775764839.000000000, ctime 1775764839.000000000
00010000:00010000:1.0:1775764839.777024:0:71579:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f14c2153600/0x23cab7dad48e6040 lrc: 2/0,0 mode: PR/PR res: [0x200000405:0x79:0x0].0x0 bits 0x1a/0x0 rrc: 3 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x929304787232d3b expref: -99 pid: 71579 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0
00000080:00200000:1.0:1775764839.777070:0:71579:0:(vvp_object.c:117:vvp_conf_set()) [0x200000405:0x79:0x0]: losing layout lock
00010000:00010000:1.0:1775764839.777207:0:71579:0:(ldlm_request.c:1438:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f14c2153600/0x23cab7dad48e6040 lrc: 1/0,0 mode: --/PR res: [0x200000405:0x79:0x0].0x0 bits 0x1a/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x929304787232d3b expref: -99 pid: 71579 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0
00010000:00010000:1.0:1775764839.777216:0:71579:0:(ldlm_request.c:1490:ldlm_cancel_pack()) 1 locks packed
00010000:00010000:1.0:1775764839.777219:0:71579:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f14c2153600/0x23cab7dad48e6040 lrc: 0/0,0 mode: --/PR res: [0x200000405:0x79:0x0].0x0 bits 0x1a/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x929304787232d3b expref: -99 pid: 71579 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0
00000020:00100000:1.0:1775764839.777246:0:71579:0:(genops.c:2442:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9f14902b3800: modify RPC slot 1 is allocated opc 36, max 7
00000100:00100000:1.0:1775764839.777255:0:71579:0:(client.c:806:ptlrpc_reassign_next_xid()) @@@ reassign xid  req@ffff9f14ab4fe300 x1862022114144000/t0(0) o36->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 488/512 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:1.0:1775764839.777809:0:71579:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fe300 pname:cluuid:pid:xid:nid:opc:job lfs:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:71579:1862022114144000:192.168.202.110@tcp:36:lfs.0
00000100:00100000:1.0:1775764839.777929:0:71579:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f149180d240 going to sleep for 50 seconds
00000100:00100000:1.0:1775764839.808539:0:2404:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2404:1862022114144256:192.168.202.110@tcp:2:lfs.0
00000100:00100000:1.0:1775764839.818870:0:71579:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1500 pname:cluuid:pid:xid:nid:opc:job lfs:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:71579:1862022114144512:192.168.202.110@tcp:36:lfs.0
00000080:00200000:1.0:1775764839.818960:0:71579:0:(vvp_io.c:1817:vvp_io_init()) [0x200000405:0x78:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0
00020000:00200000:1.0:1775764839.818975:0:71579:0:(lov_io.c:860:lov_io_iter_init()) component[0] flags 0x10
00000100:00100000:1.0:1775764839.819068:0:2403:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0]
00000100:00100000:1.0:1775764839.819082:0:2403:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ffb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2403:1862022114144640:192.168.202.110@tcp:2:lfs.0
00000100:00100000:1.0:1775764839.823151:0:2403:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ffb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2403:1862022114144640:192.168.202.110@tcp:2:lfs.0
00000100:00100000:1.0:1775764839.828127:0:44145:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1862022124504320
00000100:00100000:1.0:1775764839.828137:0:44145:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1862022124504320
00000100:00100000:1.0:1775764839.828194:0:44145:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.110@tcp, seq: 6405
00000100:00100000:1.0:1775764839.828204:0:44145:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9f14a04d3480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48+5:6487:x1862022124504320:12345-192.168.202.110@tcp:104:
00010000:00010000:1.0:1775764839.828221:0:44145:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast  ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f14c2152c00/0x23cab7dad48e6008 lrc: 2/0,0 mode: CR/CR res: [0x200000405:0x78:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x420000000000 nid: local remote: 0x929304787232ca8 expref: -99 pid: 71578 timeout: 0 lvb_type: 3 lru_score: 4 lru_type: 0
00010000:00010000:1.0:1775764839.828845:0:44145:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool
00000100:00100000:1.0:1775764839.828855:0:44145:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9f14a04d3480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48+5:6487:x1862022124504320:12345-192.168.202.110@tcp:104: Request processed in 652us (835us total) trans 0 rc 0/0
00000100:00100000:1.0:1775764839.828863:0:44145:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.110@tcp, seq: 6405
00000100:00100000:1.0:1775764839.838207:0:2396:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.110@tcp (source: 12345-192.168.202.110@tcp)
00000100:00100000:1.0:1775764839.867235:0:71579:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0000 pname:cluuid:pid:xid:nid:opc:job lfs:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:71579:1862022114144896:192.168.202.110@tcp:35:lfs.0
00000002:00100000:1.0:1775764839.867284:0:71579:0:(mdc_request.c:832:mdc_free_open()) @@@ free open request, rq_replay=0  req@ffff9f14ab4b4e00 x1862022114143232/t8589936634(8589936634) o101->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 592/680 e 0 to 0 dl 1775764894 ref 1 fl Complete:RQU/604/ffffffff rc 0/-1 job:'lfs.0' uid:0 gid:0 projid:0
00000080:00010000:1.0:1775764839.867326:0:71579:0:(file.c:6988:ll_layout_intent()) ### lustre: requeue layout lock for file [0x200000405:0x78:0x0](ffff9f14a10b7448)
00000002:00010000:1.0:1775764839.867337:0:71579:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: ,[0x200000405:0x78:0x0]) in obj [0x200000405:0x78:0x0], intent: layout flags 0
00010000:00010000:1.0:1775764839.867397:0:71579:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f14ab4c4400/0x23cab7dad48e605c lrc: 3/1,0 mode: --/CR res: [0x200000405:0x78:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 71579 initiator: MDT0 lru_score: 0 lru_type: 0
00010000:00010000:1.0:1775764839.867409:0:71579:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f14ab4c4400/0x23cab7dad48e605c lrc: 3/1,0 mode: --/CR res: [0x200000405:0x78:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x0 pid: 71579 initiator: MDT0 lru_score: 0 lru_type: 0
00010000:00010000:1.0:1775764839.867424:0:71579:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f14ab4c4400/0x23cab7dad48e605c lrc: 3/1,0 mode: --/CR res: [0x200000405:0x78:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x0 pid: 71579 initiator: MDT0 lru_score: 0 lru_type: 0
00000100:00100000:1.0:1775764839.867446:0:71579:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job lfs:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:71579:1862022114145280:192.168.202.110@tcp:101:lfs.0
00000100:00100000:1.0:1775764839.867541:0:71579:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f149180d600 going to sleep for 50 seconds
00000100:00100000:1.0:1775764839.873760:0:71579:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job lfs:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:71579:1862022114145280:192.168.202.110@tcp:101:lfs.0
00010000:00010000:1.0:1775764839.873793:0:71579:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f14ab4c4400/0x23cab7dad48e605c lrc: 4/1,0 mode: CR/CR res: [0x200000405:0x78:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x929304787232d8f expref: -99 pid: 71579 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0
00010000:00010000:1.0:1775764839.873809:0:71579:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f14ab4c4400/0x23cab7dad48e605c lrc: 4/1,0 mode: CR/CR res: [0x200000405:0x78:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x929304787232d8f expref: -99 pid: 71579 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0
00000002:00100000:1.0:1775764839.873823:0:71579:0:(mdc_locks.c:833:mdc_finish_enqueue()) @@@ op=400 disposition=1, status=0  req@ffff9f14ab4fca80 x1862022114145280/t0(0) o101->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 376/600 e 0 to 0 dl 1775764894 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'lfs.0' uid:0 gid:0 projid:0
00000002:00010000:1.0:1775764839.873835:0:71579:0:(mdc_locks.c:937:mdc_finish_enqueue()) ### layout lock returned by: layout, lvb_len: 256 ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f14ab4c4400/0x23cab7dad48e605c lrc: 3/1,0 mode: CR/CR res: [0x200000405:0x78:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x929304787232d8f expref: -99 pid: 71579 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0
00000002:00010000:1.0:1775764839.873846:0:71579:0:(mdc_locks.c:1357:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f14ab4c4400/0x23cab7dad48e605c lrc: 3/1,0 mode: CR/CR res: [0x200000405:0x78:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x929304787232d8f expref: -99 pid: 71579 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0
00010000:00010000:1.0:1775764839.873857:0:71579:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9f1494571c00 type 13 mode 16 res 8589935621/120 (0 0)
00000080:00200000:1.0:1775764839.873864:0:71579:0:(namei.c:953:ll_intent_lock()) intent lock 1024 on i1 [0x200000405:0x78:0x0] suppgids 0 0: rc 0
00000080:00010000:1.0:1775764839.873881:0:71579:0:(llite_internal.h:1984:ll_set_lock_data()) setting l_data to inode [0x200000405:0x78:0x0](ffff9f14a10b7448) for lock 0x23cab7dad48e605c
00000080:00010000:1.0:1775764839.873889:0:71579:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x23cab7dad48e605c from it ffffb06c01c8fbc8
00010000:00010000:1.0:1775764839.873910:0:71579:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f14ab4c4400/0x23cab7dad48e605c lrc: 3/1,0 mode: CR/CR res: [0x200000405:0x78:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x929304787232d8f expref: -99 pid: 71579 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0
00010000:00010000:1.0:1775764839.873927:0:71579:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f14ab4c4400/0x23cab7dad48e605c lrc: 2/0,0 mode: CR/CR res: [0x200000405:0x78:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x929304787232d8f expref: -99 pid: 71579 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0
00010000:00010000:1.0:1775764839.873951:0:71579:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f14ab4c4400/0x23cab7dad48e605c lrc: 2/1,0 mode: CR/CR res: [0x200000405:0x78:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x929304787232d8f expref: -99 pid: 71579 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0
00010000:00010000:1.0:1775764839.873962:0:71579:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f14ab4c4400/0x23cab7dad48e605c lrc: 2/1,0 mode: CR/CR res: [0x200000405:0x78:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x929304787232d8f expref: -99 pid: 71579 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0
00000080:00010000:1.0:1775764839.873970:0:71579:0:(file.c:6871:ll_layout_lock_set()) ### file [0x200000405:0x78:0x0](0000000088e74192) being reconfigured ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f14ab4c4400/0x23cab7dad48e605c lrc: 3/1,0 mode: CR/CR res: [0x200000405:0x78:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x929304787232d8f expref: -99 pid: 71579 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0
00010000:00010000:1.0:1775764839.874057:0:71579:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f14ab4c4400/0x23cab7dad48e605c lrc: 3/1,0 mode: CR/CR res: [0x200000405:0x78:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x929304787232d8f expref: -99 pid: 71579 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0
00010000:00010000:1.0:1775764839.874066:0:71579:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f14ab4c4400/0x23cab7dad48e605c lrc: 2/0,0 mode: CR/CR res: [0x200000405:0x78:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x929304787232d8f expref: -99 pid: 71579 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0
00000080:00200000:1.0:1775764839.874098:0:71579:0:(vvp_io.c:1817:vvp_io_init()) [0x200000405:0x78:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0
00020000:00200000:1.0:1775764839.874110:0:71579:0:(lov_io.c:860:lov_io_iter_init()) component[0] flags 0x10
00000080:00200000:1.0:1775764839.874171:0:71579:0:(vvp_io.c:288:vvp_io_fini()) [0x200000405:0x78:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0
00010000:00010000:1.0:1775764839.874196:0:71579:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f14ab4c4400/0x23cab7dad48e605c lrc: 2/1,0 mode: CR/CR res: [0x200000405:0x78:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x929304787232d8f expref: -99 pid: 71579 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0
00010000:00010000:1.0:1775764839.874206:0:71579:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f14ab4c4400/0x23cab7dad48e605c lrc: 2/1,0 mode: CR/CR res: [0x200000405:0x78:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x929304787232d8f expref: -99 pid: 71579 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0
00000080:00010000:1.0:1775764839.874219:0:71579:0:(file.c:6871:ll_layout_lock_set()) ### file [0x200000405:0x78:0x0](0000000088e74192) being reconfigured ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f14ab4c4400/0x23cab7dad48e605c lrc: 3/1,0 mode: CR/CR res: [0x200000405:0x78:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x929304787232d8f expref: -99 pid: 71579 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0
00000080:00200000:1.0:1775764839.874252:0:71579:0:(vvp_io.c:1817:vvp_io_init()) [0x200000405:0x78:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0
00020000:00200000:1.0:1775764839.874257:0:71579:0:(lov_io.c:860:lov_io_iter_init()) component[0] flags 0x10
00000080:00200000:1.0:1775764839.874293:0:71579:0:(vvp_io.c:288:vvp_io_fini()) [0x200000405:0x78:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0
00000008:00010000:1.0:1775764839.874312:0:71579:0:(osc_object.c:202:osc_object_ast_clear()) ### update lock size 0 blocks 0 [cma]time: 0 0 0 by oinfo size 0 blocks 0 [cma]time 1775764839 1775764839 1775764839 ns: lustre-OST0001-osc-ffff9f14902b3800 lock: 00000000255d8ffe/0x23cab7dad48e601d lrc: 1/0,0 mode: PR/PR res: [0x2c0000401:0xb7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800020000000000 nid: local remote: 0x929304787232ce0 expref: -99 pid: 71579 timeout: 0 lvb_type: 1 lru_score: 8 lru_type: 1
00000080:00200000:1.0:1775764839.876788:0:71579:0:(file.c:6766:ll_layout_conf()) [0x200000405:0x78:0x0]: layout version change: 4294967294 -> 1
00010000:00010000:1.0:1775764839.876811:0:71579:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f14ab4c4400/0x23cab7dad48e605c lrc: 3/1,0 mode: CR/CR res: [0x200000405:0x78:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x929304787232d8f expref: -99 pid: 71579 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0
00010000:00010000:1.0:1775764839.876824:0:71579:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f14ab4c4400/0x23cab7dad48e605c lrc: 2/0,0 mode: CR/CR res: [0x200000405:0x78:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x929304787232d8f expref: -99 pid: 71579 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0
00000080:00200008:1.0:1775764839.876862:0:71579:0:(file.c:447:ll_file_release()) START file f206.sanity-flr:[0x200000405:0x78:0x0](ffff9f14a10b7448), flags 1154002
00000002:00100000:1.0:1775764839.876933:0:71579:0:(mdc_request.c:964:mdc_close()) @@@ matched open  req@ffff9f14a04d0e00 x1862022114141440/t8589936630(8589936630) o101->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 592/680 e 0 to 0 dl 1775764894 ref 1 fl Complete:RPQU/604/ffffffff rc 0/-1 job:'lfs.0' uid:0 gid:0 projid:0
00000020:00100000:1.0:1775764839.876945:0:71579:0:(genops.c:2442:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9f14902b3800: modify RPC slot 1 is allocated opc 35, max 7
00000100:00100000:1.0:1775764839.876950:0:71579:0:(client.c:806:ptlrpc_reassign_next_xid()) @@@ reassign xid  req@ffff9f14ab4fca80 x1862022114145536/t0(0) o35->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:23/10 lens 392/2128 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0
00000100:00100000:1.0:1775764839.876971:0:71579:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job lfs:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:71579:1862022114145536:192.168.202.110@tcp:35:lfs.0
00000100:00100000:1.0:1775764839.877049:0:71579:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f149180d600 going to sleep for 11 seconds
00000100:00100000:1.0:1775764839.880471:0:71579:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job lfs:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:71579:1862022114145536:192.168.202.110@tcp:35:lfs.0
00000002:00100000:1.0:1775764839.880508:0:71579:0:(mdc_request.c:832:mdc_free_open()) @@@ free open request, rq_replay=0  req@ffff9f14a04d0e00 x1862022114141440/t8589936630(8589936630) o101->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 592/680 e 0 to 0 dl 1775764894 ref 1 fl Complete:RQU/604/ffffffff rc 0/-1 job:'lfs.0' uid:0 gid:0 projid:0
00000080:00200008:1.0:1775764839.881502:0:71579:0:(file.c:447:ll_file_release()) START file .:VOLATILE:0000:5037705A:fd=03:[0x200000405:0x79:0x0](ffff9f14a11100c8), flags 554001
00000002:00100000:1.0:1775764839.881602:0:71579:0:(mdc_request.c:964:mdc_close()) @@@ matched open  req@ffff9f14a04d3100 x1862022114141824/t8589936631(8589936631) o101->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 552/608 e 0 to 0 dl 1775764894 ref 1 fl Complete:RPQU/604/ffffffff rc 0/-1 job:'lfs.0' uid:0 gid:0 projid:0
00000020:00100000:1.0:1775764839.881615:0:71579:0:(genops.c:2442:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9f14902b3800: modify RPC slot 1 is allocated opc 35, max 7
00000100:00100000:1.0:1775764839.881620:0:71579:0:(client.c:806:ptlrpc_reassign_next_xid()) @@@ reassign xid  req@ffff9f14ab4fca80 x1862022114145792/t0(0) o35->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:23/10 lens 392/2128 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0
00000100:00100000:1.0:1775764839.881645:0:71579:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job lfs:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:71579:1862022114145792:192.168.202.110@tcp:35:lfs.0
00000100:00100000:1.0:1775764839.881732:0:71579:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f149180d600 going to sleep for 11 seconds
00000100:00100000:1.0:1775764839.943946:0:71582:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4b6a00 pname:cluuid:pid:xid:nid:opc:job lfs:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:71582:1862022114146560:192.168.202.110@tcp:101:lfs.0
00010000:00010000:1.0:1775764839.943970:0:71582:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f14c2153400/0x23cab7dad48e6071 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x929304787232dea expref: -99 pid: 71582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0
00010000:00010000:1.0:1775764839.943986:0:71582:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f14c2153400/0x23cab7dad48e6071 lrc: 4/1,0 mode: --/PR res: [0x200000405:0x78:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x929304787232dea expref: -99 pid: 71582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0
00010000:00010000:1.0:1775764839.943993:0:71582:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f14c2153400/0x23cab7dad48e6071 lrc: 4/1,0 mode: --/PR res: [0x200000405:0x78:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x929304787232dea expref: -99 pid: 71582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0
00010000:00010000:1.0:1775764839.944004:0:71582:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f14c2153400/0x23cab7dad48e6071 lrc: 4/1,0 mode: PR/PR res: [0x200000405:0x78:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x929304787232dea expref: -99 pid: 71582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0
00010000:00010000:1.0:1775764839.944013:0:71582:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f14c2153400/0x23cab7dad48e6071 lrc: 4/1,0 mode: PR/PR res: [0x200000405:0x78:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x929304787232dea expref: -99 pid: 71582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0
00010000:00010000:1.0:1775764839.944025:0:71582:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f14c2153400/0x23cab7dad48e6071 lrc: 5/2,0 mode: PR/PR res: [0x200000405:0x78:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x929304787232dea expref: -99 pid: 71582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0
00010000:00010000:1.0:1775764839.944031:0:71582:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f14c2153400/0x23cab7dad48e6071 lrc: 5/2,0 mode: PR/PR res: [0x200000405:0x78:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x929304787232dea expref: -99 pid: 71582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0
00010000:00010000:1.0:1775764839.944037:0:71582:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f14c2153400/0x23cab7dad48e6071 lrc: 4/1,0 mode: PR/PR res: [0x200000405:0x78:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x929304787232dea expref: -99 pid: 71582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0
00000002:00100000:1.0:1775764839.944046:0:71582:0:(mdc_locks.c:833:mdc_finish_enqueue()) @@@ op=1 disposition=2b, status=0  req@ffff9f14ab4b6a00 x1862022114146560/t8589936642(8589936642) o101->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 2160/864 e 0 to 0 dl 1775764894 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'lfs.0' uid:0 gid:0 projid:0
00000002:00100000:1.0:1775764839.944058:0:71582:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data  req@ffff9f14ab4b6a00 x1862022114146560/t8589936642(8589936642) o101->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 2160/864 e 0 to 0 dl 1775764894 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'lfs.0' uid:0 gid:0 projid:0
00000002:00010000:1.0:1775764839.944070:0:71582:0:(mdc_locks.c:937:mdc_finish_enqueue()) ### layout lock returned by: open, lvb_len: 256 ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f14c2153400/0x23cab7dad48e6071 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x78:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x929304787232dea expref: -99 pid: 71582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0
00000002:00010000:1.0:1775764839.944080:0:71582:0:(mdc_locks.c:1357:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f14c2153400/0x23cab7dad48e6071 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x78:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x929304787232dea expref: -99 pid: 71582 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0
00010000:00010000:1.0:1775764839.944088:0:71582:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9f1494571c00 type 13 mode 4 res 8589935621/120 (0 0)
00000080:00200000:1.0:1775764839.944094:0:71582:0:(namei.c:953:ll_intent_lock()) intent lock 1 on i1 [0x200000007:0x1:0x0] suppgids 0 -1: rc 0
00000080:00200000:1.0:1775764839.944189:0:71582:0:(llite_lib.c:2941:ll_update_inode()) inode=[0x200000405:0x78:0x0], updating i_size 0
00000080:00200000:1.0:1775764839.944193:0:71582:0:(namei.c:147:ll_iget()) got inode: [0x200000405:0x78:0x0](ffff9f14a10b7448): rc = 0
00000080:00200000:1.0:1775764839.944213:0:71582:0:(file.c:6766:ll_layout_conf()) [0x200000405:0x78:0x0]: layout version change: 1 -> 1
00000080:00010000:1.0:1775764839.944223:0:71582:0:(llite_internal.h:1984:ll_set_lock_data()) setting l_data to inode [0x200000405:0x78:0x0](ffff9f14a10b7448) for lock 0x23cab7dad48e6071
00000080:00010000:1.0:1775764839.944234:0:71582:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000405:0x78:0x0](ffff9f14a10b7448)
00000080:00200008:1.0:1775764839.944250:0:71582:0:(file.c:1057:ll_file_open()) START file f206.sanity-flr:[0x200000405:0x78:0x0](ffff9f14a10b7448), flags 154400
00000002:00100000:1.0:1775764839.944273:0:71582:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data  req@ffff9f14ab4b6a00 x1862022114146560/t8589936642(8589936642) o101->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 784/864 e 0 to 0 dl 1775764894 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'lfs.0' uid:0 gid:0 projid:0
00000080:00010000:1.0:1775764839.944320:0:71582:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x23cab7dad48e6071 from it ffff9f1489a38060
00010000:00010000:1.0:1775764839.944325:0:71582:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f14c2153400/0x23cab7dad48e6071 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x78:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x929304787232dea expref: -99 pid: 71582 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0
00010000:00010000:1.0:1775764839.944334:0:71582:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f14c2153400/0x23cab7dad48e6071 lrc: 2/0,0 mode: PR/PR res: [0x200000405:0x78:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x929304787232dea expref: -99 pid: 71582 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0
00000080:00200000:1.0:1775764839.944356:0:71582:0:(file.c:6593:ll_inode_permission()) VFS Op:inode=[0x200000405:0x78:0x0](ffff9f14a10b7448), inode mode 81a4 mask 44
00000080:00200000:1.0:1775764839.944449:0:71582:0:(file.c:6593:ll_inode_permission()) VFS Op:inode=[0x200000405:0x78:0x0](ffff9f14a10b7448), inode mode 81a4 mask 4
00000080:00200000:1.0:1775764839.944457:0:71582:0:(xattr.c:727:ll_xattr_get()) VFS Op:inode=[0x200000405:0x78:0x0](ffff9f14a10b7448), xattr lov
00000080:00200008:1.0:1775764839.944498:0:71582:0:(file.c:6226:ll_getattr_dentry()) START file f206.sanity-flr:[0x200000405:0x78:0x0](ffff9f14a10b7448), request_mask 2047, flags 0, foreign 0
00000080:00200000:1.0:1775764839.944504:0:71582:0:(file.c:6121:ll_inode_revalidate()) VFS Op:inode=[0x200000405:0x78:0x0](ffff9f14a10b7448),name=f206.sanity-flr
00000002:00010000:1.0:1775764839.944521:0:71582:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: ,[0x200000405:0x78:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0
00010000:00010000:1.0:1775764839.944530:0:71582:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f14c2153400/0x23cab7dad48e6071 lrc: 2/1,0 mode: PR/PR res: [0x200000405:0x78:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x929304787232dea expref: -99 pid: 71582 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0
00010000:00010000:1.0:1775764839.944539:0:71582:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f14c2153400/0x23cab7dad48e6071 lrc: 2/1,0 mode: PR/PR res: [0x200000405:0x78:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x929304787232dea expref: -99 pid: 71582 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0
00000080:00200000:1.0:1775764839.944546:0:71582:0:(namei.c:953:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1
00000080:00010000:1.0:1775764839.944550:0:71582:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000405:0x78:0x0](ffff9f14a10b7448)
00000080:00010000:1.0:1775764839.944582:0:71582:0:(llite_internal.h:1984:ll_set_lock_data()) setting l_data to inode [0x200000405:0x78:0x0](ffff9f14a10b7448) for lock 0x23cab7dad48e6071
00000080:00010000:1.0:1775764839.944586:0:71582:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x23cab7dad48e6071 from it ffffb06c01ce7cd8
00010000:00010000:1.0:1775764839.944589:0:71582:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f14c2153400/0x23cab7dad48e6071 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x78:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x929304787232dea expref: -99 pid: 71582 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0
00010000:00010000:1.0:1775764839.944596:0:71582:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f14c2153400/0x23cab7dad48e6071 lrc: 2/0,0 mode: PR/PR res: [0x200000405:0x78:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x929304787232dea expref: -99 pid: 71582 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0
00000080:00200008:1.0:1775764839.944645:0:71582:0:(file.c:447:ll_file_release()) START file f206.sanity-flr:[0x200000405:0x78:0x0](ffff9f14a10b7448), flags 154000
00000002:00100000:1.0:1775764839.944686:0:71582:0:(mdc_request.c:964:mdc_close()) @@@ matched open  req@ffff9f14ab4b6a00 x1862022114146560/t8589936642(8589936642) o101->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 784/864 e 0 to 0 dl 1775764894 ref 1 fl Complete:RPQU/604/ffffffff rc 0/-1 job:'lfs.0' uid:0 gid:0 projid:0
00000020:00100000:1.0:1775764839.944696:0:71582:0:(genops.c:2442:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9f14902b3800: modify RPC slot 1 is allocated opc 35, max 7
00000100:00100000:1.0:1775764839.944701:0:71582:0:(client.c:806:ptlrpc_reassign_next_xid()) @@@ reassign xid  req@ffff9f14ab4ff800 x1862022114146816/t0(0) o35->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:23/10 lens 392/2128 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0
00000100:00100000:1.0:1775764839.944722:0:71582:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff800 pname:cluuid:pid:xid:nid:opc:job lfs:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:71582:1862022114146816:192.168.202.110@tcp:35:lfs.0
00000100:00100000:1.0:1775764839.944799:0:71582:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f149180d600 going to sleep for 11 seconds
00000100:00100000:1.0:1775764839.951601:0:44145:0:(service.c:153:ptlrpc_grow_req_bufs()) ldlm_cbd: allocate 1 new 8192-byte reqbufs (0/2 left), rc = 0
00000100:00100000:1.0:1775764839.951622:0:44145:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1862022124505088
00000100:00100000:1.0:1775764839.951632:0:44145:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1862022124505088
00000100:00100000:1.0:1775764839.951684:0:44145:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.110@tcp, seq: 6407
00000100:00100000:1.0:1775764839.951694:0:44145:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9f14902ccc50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48+5:7675:x1862022124505088:12345-192.168.202.110@tcp:104:
00010000:00010000:1.0:1775764839.951713:0:44145:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast  ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f14c2153400/0x23cab7dad48e6071 lrc: 2/0,0 mode: PR/PR res: [0x200000405:0x78:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x420000000000 nid: local remote: 0x929304787232dea expref: -99 pid: 71582 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0
00010000:00010000:1.0:1775764839.951778:0:44145:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool
00000100:00100000:1.0:1775764839.951784:0:44145:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9f14902ccc50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48+5:7675:x1862022124505088:12345-192.168.202.110@tcp:104: Request processed in 92us (353us total) trans 0 rc 0/0
00000100:00100000:1.0:1775764839.951792:0:44145:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.110@tcp, seq: 6407
00000100:00100000:1.0:1775764839.956720:0:71582:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1491686680 pname:cluuid:pid:xid:nid:opc:job lfs:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:71582:1862022114147072:192.168.202.110@tcp:101:lfs.0
00010000:00010000:1.0:1775764839.956746:0:71582:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode CW ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f149153f200/0x23cab7dad48e6078 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x929304787232dff expref: -99 pid: 71582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0
00010000:00010000:1.0:1775764839.956764:0:71582:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f149153f200/0x23cab7dad48e6078 lrc: 4/1,0 mode: --/CW res: [0x200000405:0x78:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x929304787232dff expref: -99 pid: 71582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0
00010000:00010000:1.0:1775764839.956773:0:71582:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f149153f200/0x23cab7dad48e6078 lrc: 4/1,0 mode: --/CW res: [0x200000405:0x78:0x0].0x0 bits 0x5/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x929304787232dff expref: -99 pid: 71582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0
00010000:00010000:1.0:1775764839.956784:0:71582:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f149153f200/0x23cab7dad48e6078 lrc: 4/1,0 mode: CW/CW res: [0x200000405:0x78:0x0].0x0 bits 0x5/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x929304787232dff expref: -99 pid: 71582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0
00010000:00010000:1.0:1775764839.956795:0:71582:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f149153f200/0x23cab7dad48e6078 lrc: 4/1,0 mode: CW/CW res: [0x200000405:0x78:0x0].0x0 bits 0x5/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x929304787232dff expref: -99 pid: 71582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0
00010000:00010000:1.0:1775764839.956807:0:71582:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f149153f200/0x23cab7dad48e6078 lrc: 5/1,1 mode: CW/CW res: [0x200000405:0x78:0x0].0x0 bits 0x5/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x929304787232dff expref: -99 pid: 71582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0
00010000:00010000:1.0:1775764839.956816:0:71582:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f149153f200/0x23cab7dad48e6078 lrc: 5/1,1 mode: CW/CW res: [0x200000405:0x78:0x0].0x0 bits 0x5/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x929304787232dff expref: -99 pid: 71582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0
00010000:00010000:1.0:1775764839.956823:0:71582:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f149153f200/0x23cab7dad48e6078 lrc: 4/0,1 mode: CW/CW res: [0x200000405:0x78:0x0].0x0 bits 0x5/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x929304787232dff expref: -99 pid: 71582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0
00000002:00100000:1.0:1775764839.956834:0:71582:0:(mdc_locks.c:833:mdc_finish_enqueue()) @@@ op=1 disposition=200002b, status=0  req@ffff9f1491686680 x1862022114147072/t8589936644(8589936644) o101->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 2152/864 e 0 to 0 dl 1775764894 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'lfs.0' uid:0 gid:0 projid:0
00000002:00100000:1.0:1775764839.956849:0:71582:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data  req@ffff9f1491686680 x1862022114147072/t8589936644(8589936644) o101->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 2152/864 e 0 to 0 dl 1775764894 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'lfs.0' uid:0 gid:0 projid:0
00000002:00010000:1.0:1775764839.956868:0:71582:0:(mdc_locks.c:1357:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f149153f200/0x23cab7dad48e6078 lrc: 3/0,1 mode: CW/CW res: [0x200000405:0x78:0x0].0x0 bits 0x5/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x929304787232dff expref: -99 pid: 71582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0
00010000:00010000:1.0:1775764839.956877:0:71582:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9f1494571c00 type 13 mode 8 res 8589935621/120 (0 0)
00000080:00200000:1.0:1775764839.956885:0:71582:0:(namei.c:953:ll_intent_lock()) intent lock 1 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0
00000080:00200000:1.0:1775764839.956991:0:71582:0:(llite_lib.c:2941:ll_update_inode()) inode=[0x200000405:0x78:0x0], updating i_size 0
00000080:00010000:1.0:1775764839.956998:0:71582:0:(llite_internal.h:1984:ll_set_lock_data()) setting l_data to inode [0x200000405:0x78:0x0](ffff9f14a10b7448) for lock 0x23cab7dad48e6078
00000080:00010000:1.0:1775764839.957005:0:71582:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x23cab7dad48e6078 from it ffffb06c01ce7bf0
00010000:00010000:1.0:1775764839.957008:0:71582:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f149153f200/0x23cab7dad48e6078 lrc: 3/0,1 mode: CW/CW res: [0x200000405:0x78:0x0].0x0 bits 0x5/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x929304787232dff expref: -99 pid: 71582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0
00010000:00010000:1.0:1775764839.957016:0:71582:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f149153f200/0x23cab7dad48e6078 lrc: 2/0,0 mode: CW/CW res: [0x200000405:0x78:0x0].0x0 bits 0x5/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x929304787232dff expref: -99 pid: 71582 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0
00000002:00100000:1.0:1775764839.957049:0:71582:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data  req@ffff9f1491686680 x1862022114147072/t8589936644(8589936644) o101->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 776/864 e 0 to 0 dl 1775764894 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'lfs.0' uid:0 gid:0 projid:0
00000080:00200000:1.0:1775764839.957147:0:71582:0:(file.c:6593:ll_inode_permission()) VFS Op:inode=[0x200000405:0x78:0x0](ffff9f14a10b7448), inode mode 81a4 mask 4
00000080:00200000:1.0:1775764839.957155:0:71582:0:(xattr.c:727:ll_xattr_get()) VFS Op:inode=[0x200000405:0x78:0x0](ffff9f14a10b7448), xattr lov
00000080:00200008:1.0:1775764839.957194:0:71582:0:(file.c:6226:ll_getattr_dentry()) START file f206.sanity-flr:[0x200000405:0x78:0x0](ffff9f14a10b7448), request_mask 2047, flags 0, foreign 0
00000080:00200000:1.0:1775764839.957201:0:71582:0:(file.c:6121:ll_inode_revalidate()) VFS Op:inode=[0x200000405:0x78:0x0](ffff9f14a10b7448),name=f206.sanity-flr
00000002:00010000:1.0:1775764839.957214:0:71582:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: ,[0x200000405:0x78:0x0]) in obj [0x200000405:0x78:0x0], intent: getattr flags 0
00010000:00010000:1.0:1775764839.957221:0:71582:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9f1494571c00 type 13 mode 30 res 8589935621/120 (0 0)
00010000:00010000:1.0:1775764839.957261:0:71582:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f14ab4c5a00/0x23cab7dad48e607f lrc: 3/1,0 mode: --/CR res: [0x200000405:0x78:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 71582 initiator: MDT0 lru_score: 0 lru_type: 0
00010000:00010000:1.0:1775764839.957269:0:71582:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f14ab4c5a00/0x23cab7dad48e607f lrc: 3/1,0 mode: --/CR res: [0x200000405:0x78:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 71582 initiator: MDT0 lru_score: 0 lru_type: 0
00010000:00010000:1.0:1775764839.957280:0:71582:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f14ab4c5a00/0x23cab7dad48e607f lrc: 3/1,0 mode: --/CR res: [0x200000405:0x78:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 71582 initiator: MDT0 lru_score: 0 lru_type: 0
00000100:00100000:1.0:1775764839.957304:0:71582:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff480 pname:cluuid:pid:xid:nid:opc:job lfs:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:71582:1862022114147328:192.168.202.110@tcp:101:lfs.0
00000100:00100000:1.0:1775764839.957968:0:71582:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f149180d0c0 going to sleep for 50 seconds
00000100:00100000:1.0:1775764839.961818:0:2396:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.110@tcp (source: 12345-192.168.202.110@tcp)
00000100:00100000:1.0:1775764839.972954:0:2396:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.110@tcp (source: 12345-192.168.202.110@tcp)
00000100:00100000:1.0:1775764839.981784:0:2395:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.110@tcp (source: 12345-192.168.202.110@tcp)
00000100:00100000:1.0:1775764839.981816:0:2395:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.110@tcp (source: 12345-192.168.202.110@tcp)
00000100:00100000:1.0:1775764839.981843:0:44145:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1862022124505472
00000100:00100000:1.0:1775764839.981852:0:44145:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1862022124505472
00000100:00100000:1.0:1775764839.981901:0:44145:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1862022124505600
00000100:00100000:1.0:1775764839.981904:0:44145:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1862022124505600
00000100:00100000:1.0:1775764839.981916:0:44145:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.110@tcp, seq: 6410
00000100:00100000:1.0:1775764839.981924:0:44145:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9f14ab4fce00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48+6:15081:x1862022124505472:12345-192.168.202.110@tcp:104:
00010000:00010000:1.0:1775764839.981943:0:44145:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast  ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f14ab4c5a00/0x23cab7dad48e607f lrc: 2/0,0 mode: PR/PR res: [0x200000405:0x78:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x420000000000 nid: local remote: 0x929304787232e14 expref: -99 pid: 71582 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0
00010000:00010000:1.0:1775764839.982013:0:44145:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool
00000100:00100000:1.0:1775764839.982019:0:44145:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9f14ab4fce00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48+6:15081:x1862022124505472:12345-192.168.202.110@tcp:104: Request processed in 95us (243us total) trans 0 rc 0/0
00000100:00100000:1.0:1775764839.982025:0:44145:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.110@tcp, seq: 6410
00000100:00100000:1.0:1775764839.982042:0:44145:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.110@tcp, seq: 6411
00000100:00100000:1.0:1775764839.982046:0:44145:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9f14ab4fdf80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48+5:15081:x1862022124505600:12345-192.168.202.110@tcp:104:
00010000:00010000:1.0:1775764839.982053:0:44145:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast  ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f14ab4c4400/0x23cab7dad48e605c lrc: 2/0,0 mode: CR/CR res: [0x200000405:0x78:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x420000000000 nid: local remote: 0x929304787232d8f expref: -99 pid: 71579 timeout: 0 lvb_type: 3 lru_score: 4 lru_type: 0
00010000:00010000:1.0:1775764839.982089:0:44145:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to priority blp list, 2 blwis in pool
00000100:00100000:1.0:1775764839.982094:0:44145:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9f14ab4fdf80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48+5:15081:x1862022124505600:12345-192.168.202.110@tcp:104: Request processed in 47us (277us total) trans 0 rc 0/0
00000100:00100000:1.0:1775764839.982098:0:44145:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.110@tcp, seq: 6411
00000080:00200000:1.0:1775764840.007742:0:71583:0:(file.c:6121:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9f1491deba88),name=/
00000002:00010000:1.0:1775764840.007770:0:71583:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0
00010000:00010000:1.0:1775764840.007788:0:71583:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f14c2152c00/0x23cab7dad48e6063 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x929304787232da4 expref: -99 pid: 71580 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 0
00010000:00010000:1.0:1775764840.007804:0:71583:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f14c2152c00/0x23cab7dad48e6063 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x929304787232da4 expref: -99 pid: 71580 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 0
00000080:00200000:1.0:1775764840.007813:0:71583:0:(namei.c:953:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1
00000080:00010000:1.0:1775764840.007821:0:71583:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9f1491deba88)
00000080:00010000:1.0:1775764840.007825:0:71583:0:(llite_internal.h:1984:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9f1491deba88) for lock 0x23cab7dad48e6063
00000080:00010000:1.0:1775764840.007832:0:71583:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x23cab7dad48e6063 from it ffffb06c01067bf8
00010000:00010000:1.0:1775764840.007836:0:71583:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f14c2152c00/0x23cab7dad48e6063 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x929304787232da4 expref: -99 pid: 71580 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 0
00010000:00010000:1.0:1775764840.007844:0:71583:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f14c2152c00/0x23cab7dad48e6063 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x929304787232da4 expref: -99 pid: 71580 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 0
00000080:00200000:1.0:1775764840.007860:0:71583:0:(file.c:6593:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9f1491deba88), inode mode 41ed mask 1
00000080:00200000:1.0:1775764840.007874:0:71583:0:(namei.c:1409:ll_atomic_open()) VFS Op:name=f206.sanity-flr, dir=[0x200000007:0x1:0x0](ffff9f1491deba88), file ffff9f1491538480, open_flags 18800, mode 0 opened 0
00000080:00200000:1.0:1775764840.007880:0:71583:0:(namei.c:1073:ll_lookup_it()) VFS Op:name=f206.sanity-flr, dir=[0x200000007:0x1:0x0](ffff9f1491deba88), intent=open
00000002:00010000:1.0:1775764840.007905:0:71583:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: f206.sanity-flr,[0x0:0x0:0x0]) in obj [0x200000007:0x1:0x0], intent: open flags 0304001
00010000:00010000:1.0:1775764840.007957:0:71583:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f14ab4c5400/0x23cab7dad48e6086 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 71583 initiator: MDT0 lru_score: 0 lru_type: 0
00010000:00010000:1.0:1775764840.007965:0:71583:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f14ab4c5400/0x23cab7dad48e6086 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x0 pid: 71583 initiator: MDT0 lru_score: 0 lru_type: 0
00000020:00100000:1.0:1775764840.007977:0:71583:0:(genops.c:2442:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9f14902b3800: modify RPC slot 1 is allocated opc 101, max 7
00000100:00100000:1.0:1775764840.007989:0:71583:0:(client.c:806:ptlrpc_reassign_next_xid()) @@@ reassign xid  req@ffff9f14ab4fe680 x1862022114148736/t0(0) o101->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 2160/133360 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0
00010000:00010000:1.0:1775764840.007999:0:71583:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f14ab4c5400/0x23cab7dad48e6086 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x0 pid: 71583 initiator: MDT0 lru_score: 0 lru_type: 0
00000100:00100000:1.0:1775764840.008019:0:71583:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fe680 pname:cluuid:pid:xid:nid:opc:job lfs:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:71583:1862022114148736:192.168.202.110@tcp:101:lfs.0
00000100:00100000:1.0:1775764840.008238:0:71583:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f149180d9c0 going to sleep for 50 seconds
00000100:00100000:1.0:1775764841.994775:0:2403:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2403:1862022114148992:192.168.202.110@tcp:400:kworker.0
00000100:00100000:1.0:1775764841.994869:0:2403:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2403:1862022114149248:192.168.202.110@tcp:400:kworker.0
00000100:00100000:1.0:1775764841.994911:0:2403:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fe680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2403:1862022114149504:192.168.202.110@tcp:400:kworker.0
00000100:00100000:1.0:1775764841.994936:0:2403:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [1->0]
00000100:00100000:1.0:1775764841.994941:0:2403:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fc700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2403:1862022114149120:192.168.202.110@tcp:400:kworker.0
00000100:00100000:1.0:1775764841.994959:0:2403:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2403:1862022114149376:192.168.202.110@tcp:400:kworker.0
00000100:00100000:1.0:1775764841.999947:0:2403:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fc700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2403:1862022114149120:192.168.202.110@tcp:400:kworker.0
00000100:00100000:1.0:1775764841.999963:0:2403:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2403:1862022114149376:192.168.202.110@tcp:400:kworker.0
00000100:00100000:1.0:1775764841.999972:0:2403:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2403:1862022114148992:192.168.202.110@tcp:400:kworker.0
00000100:00100000:1.0:1775764841.999983:0:2403:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2403:1862022114149248:192.168.202.110@tcp:400:kworker.0
00000100:00100000:1.0:1775764841.999992:0:2403:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fe680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2403:1862022114149504:192.168.202.110@tcp:400:kworker.0
00000100:00100000:1.0:1775764843.001298:0:44145:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1862022124506112
00000100:00100000:1.0:1775764843.001304:0:44145:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1862022124506112
00000100:00100000:1.0:1775764843.001359:0:44145:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.110@tcp, seq: 6412
00000100:00100000:1.0:1775764843.001369:0:44145:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9f14a04d3100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41+3:42782:x1862022124506112:12345-192.168.202.110@tcp:104:
00010000:00010000:1.0:1775764843.001387:0:44145:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast  ns: MGC192.168.202.110@tcp lock: ffff9f149153f600/0x23cab7dad48e5fe5 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x1400000000000 nid: local remote: 0x929304787232c4d expref: -99 pid: 39317 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0
00010000:00010000:1.0:1775764843.001463:0:44145:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool
00000100:00100000:1.0:1775764843.001471:0:44145:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9f14a04d3100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41+3:42782:x1862022124506112:12345-192.168.202.110@tcp:104: Request processed in 103us (313us total) trans 0 rc 0/0
00000100:00100000:1.0:1775764843.001478:0:44145:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.110@tcp, seq: 6412
10000000:01000000:1.0:1775764844.237459:0:39317:0:(mgc_request.c:611:do_requeue()) updating log lustre-client
10000000:01000000:1.0:1775764844.237467:0:39317:0:(mgc_request.c:1819:mgc_process_log()) Process log lustre-client-ffff9f14902b3800 from 79
10000000:01000000:1.0:1775764844.237472:0:39317:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-client (res 0x65727473756c)
00010000:00010000:1.0:1775764844.237527:0:39317:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.202.110@tcp lock: ffff9f14ab4c5c00/0x23cab7dad48e608d lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 39317 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0
00010000:00010000:1.0:1775764844.237538:0:39317:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.202.110@tcp lock: ffff9f14ab4c5c00/0x23cab7dad48e608d lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 39317 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0
00010000:00010000:1.0:1775764844.237551:0:39317:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.202.110@tcp lock: ffff9f14ab4c5c00/0x23cab7dad48e608d lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 39317 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0
00000100:00100000:1.0:1775764844.237598:0:39317:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fe680 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:39317:1862022114149888:192.168.202.110@tcp:101:ll_cfg_requeue.0
00000100:00100000:1.0:1775764844.237697:0:39317:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f149180db40 going to sleep for 11 seconds
00000100:00100000:1.0:1775764844.243082:0:39317:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fe680 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:39317:1862022114149888:192.168.202.110@tcp:101:ll_cfg_requeue.0
00010000:00010000:1.0:1775764844.243114:0:39317:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGC192.168.202.110@tcp lock: ffff9f14ab4c5c00/0x23cab7dad48e608d lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x929304787232e5a expref: -99 pid: 39317 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0
00010000:00010000:1.0:1775764844.243128:0:39317:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: MGC192.168.202.110@tcp lock: ffff9f14ab4c5c00/0x23cab7dad48e608d lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x929304787232e5a expref: -99 pid: 39317 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0
00000100:00100000:1.0:1775764844.243397:0:39317:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fe680 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:39317:1862022114150016:192.168.202.110@tcp:501:ll_cfg_requeue.0
00000100:00100000:1.0:1775764844.243477:0:39317:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f149180db40 going to sleep for 11 seconds
00000100:00100000:1.0:1775764844.250145:0:39317:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fe680 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:39317:1862022114150016:192.168.202.110@tcp:501:ll_cfg_requeue.0
00000100:00100000:1.0:1775764844.250207:0:39317:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fe680 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:39317:1862022114150144:192.168.202.110@tcp:503:ll_cfg_requeue.0
00000100:00100000:1.0:1775764844.250319:0:39317:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f149180db40 going to sleep for 11 seconds
00000100:00100000:1.0:1775764844.264399:0:39317:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fe680 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:39317:1862022114150144:192.168.202.110@tcp:503:ll_cfg_requeue.0
00000100:00100000:1.0:1775764846.181655:0:2396:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.110@tcp (source: 12345-192.168.202.110@tcp)
00000100:00100000:1.0:1775764846.181704:0:2396:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.110@tcp (source: 12345-192.168.202.110@tcp)
00000100:00100000:1.0:1775764847.114959:0:2403:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2403:1862022114151168:192.168.202.110@tcp:400:kworker.0
00000100:00100000:1.0:1775764847.115056:0:2403:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fc700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2403:1862022114151424:192.168.202.110@tcp:400:kworker.0
00000100:00100000:1.0:1775764847.115091:0:2403:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [1->0]
00000100:00100000:1.0:1775764847.115097:0:2403:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2403:1862022114151040:192.168.202.110@tcp:400:kworker.0
00000100:00100000:1.0:1775764847.115125:0:2403:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2403:1862022114151296:192.168.202.110@tcp:400:kworker.0
00000100:00100000:1.0:1775764847.115146:0:2403:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2403:1862022114151552:192.168.202.110@tcp:400:kworker.0
00000100:00100000:1.0:1775764847.117862:0:2403:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2403:1862022114151296:192.168.202.110@tcp:400:kworker.0
00000100:00100000:1.0:1775764847.117880:0:2403:0:(client.c:3082:ptlrpc_free_committed()) lustre-OST0001-osc-ffff9f14902b3800: committing for last_committed 17179874154 gen 2
00000100:00100000:1.0:1775764847.117900:0:2403:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2403:1862022114151552:192.168.202.110@tcp:400:kworker.0
00000100:00100000:1.0:1775764847.117910:0:2403:0:(client.c:3082:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9f14902b3800: committing for last_committed 8589936646 gen 1
00000100:00100000:1.0:1775764847.117931:0:2403:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2403:1862022114151168:192.168.202.110@tcp:400:kworker.0
00000100:00100000:1.0:1775764847.117940:0:2403:0:(client.c:3082:ptlrpc_free_committed()) lustre-OST0000-osc-ffff9f14902b3800: committing for last_committed 21474840665 gen 1
00000100:00100000:1.0:1775764847.117947:0:2403:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fc700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2403:1862022114151424:192.168.202.110@tcp:400:kworker.0
00000020:01000000:1.0:1775764847.416391:0:39317:0:(obd_config.c:2143:class_config_parse_llog()) Processed log lustre-client gen 82-84 (rc=0)
10000000:01000000:1.0:1775764847.416414:0:39317:0:(mgc_request.c:1896:mgc_process_log()) MGC192.168.202.110@tcp: configuration from log 'lustre-client' succeeded (0).
00010000:00010000:1.0:1775764847.416425:0:39317:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.202.110@tcp lock: ffff9f1498179200/0x23cab7dad48e609b lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x929304787232f6b expref: -99 pid: 39317 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0
00010000:00010000:1.0:1775764847.416437:0:39317:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.202.110@tcp lock: ffff9f1498179200/0x23cab7dad48e609b lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x929304787232f6b expref: -99 pid: 39317 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0
10000000:01000000:1.0:1775764847.416448:0:39317:0:(mgc_request.c:611:do_requeue()) updating log lustre-sptlrpc
10000000:01000000:1.0:1775764847.416450:0:39317:0:(mgc_request.c:1819:mgc_process_log()) Process log lustre-sptlrpc-ffff9f149022cd40 from 1
10000000:01000000:1.0:1775764847.416454:0:39317:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-sptlrpc (res 0x65727473756c)
00010000:00010000:1.0:1775764847.416490:0:39317:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.202.110@tcp lock: ffff9f14ab4c4400/0x23cab7dad48e60a2 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 39317 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0
00010000:00010000:1.0:1775764847.416499:0:39317:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.202.110@tcp lock: ffff9f14ab4c4400/0x23cab7dad48e60a2 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 39317 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0
00010000:00010000:1.0:1775764847.416508:0:39317:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.202.110@tcp lock: ffff9f14ab4c4400/0x23cab7dad48e60a2 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 39317 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0
00000100:00100000:1.0:1775764847.416527:0:39317:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fc700 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:39317:1862022114152320:192.168.202.110@tcp:101:ll_cfg_requeue.0
00000100:00100000:1.0:1775764850.663652:0:2396:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.110@tcp (source: 12345-192.168.202.110@tcp)
00000100:00100000:1.0:1775764850.663673:0:2396:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.110@tcp (source: 12345-192.168.202.110@tcp)
00000100:00100000:1.0:1775764850.663696:0:44145:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1862022124525056
00000100:00100000:1.0:1775764850.663705:0:44145:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1862022124525056
00000100:00100000:1.0:1775764850.663741:0:44145:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1862022124525184
00000100:00100000:1.0:1775764850.663743:0:44145:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1862022124525184
00000100:00100000:1.0:1775764850.663752:0:44145:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.110@tcp, seq: 6415
00000100:00100000:1.0:1775764850.663758:0:44145:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9f14ab4fd880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41+4:42953:x1862022124525056:12345-192.168.202.110@tcp:104:
00010000:00010000:1.0:1775764850.663778:0:44145:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast  ns: MGC192.168.202.110@tcp lock: ffff9f14ab4c4400/0x23cab7dad48e60a2 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x1400000000000 nid: local remote: 0x929304787232f72 expref: -99 pid: 39317 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0
00010000:00010000:1.0:1775764850.663865:0:44145:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool
00000100:00100000:1.0:1775764850.663870:0:44145:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9f14ab4fd880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41+4:42953:x1862022124525056:12345-192.168.202.110@tcp:104: Request processed in 113us (227us total) trans 0 rc 0/0
00000100:00100000:1.0:1775764850.663876:0:44145:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.110@tcp, seq: 6415
00000100:00100000:1.0:1775764850.663892:0:44145:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.110@tcp, seq: 6416
00000100:00100000:1.0:1775764850.663898:0:44145:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9f14ab4ff480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41+3:42953:x1862022124525184:12345-192.168.202.110@tcp:104:
00010000:00010000:1.0:1775764850.663906:0:44145:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast  ns: MGC192.168.202.110@tcp lock: ffff9f1498179200/0x23cab7dad48e609b lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x1400000000000 nid: local remote: 0x929304787232f6b expref: -99 pid: 39317 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0
00010000:00010000:1.0:1775764850.663941:0:44145:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to priority blp list, 2 blwis in pool
00000100:00100000:1.0:1775764850.663946:0:44145:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9f14ab4ff480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41+3:42953:x1862022124525184:12345-192.168.202.110@tcp:104: Request processed in 48us (271us total) trans 0 rc 0/0
00000100:00100000:1.0:1775764850.663951:0:44145:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.110@tcp, seq: 6416
00000100:00100000:1.0:1775764851.810800:0:71647:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job llog_process_th:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:71647:1862022114153472:192.168.202.110@tcp:502:llog_process_th.0
00000100:00100000:1.0:1775764851.810946:0:71647:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f149180df00 going to sleep for 11 seconds
00000100:00100000:1.0:1775764852.236583:0:2403:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2403:1862022114153984:192.168.202.110@tcp:400:kworker.0
00000100:00100000:1.0:1775764852.236688:0:2403:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2403:1862022114154240:192.168.202.110@tcp:400:kworker.0
00010000:00010000:1.0:1775764855.052505:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00010000:00010000:1.0:1775764857.098761:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775764857.371172:0:2403:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2403:1862022114154624:192.168.202.110@tcp:400:kworker.0
00000100:00100000:1.0:1775764857.371186:0:2403:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2403:1862022114154880:192.168.202.110@tcp:400:kworker.0
00000100:00100000:1.0:1775764857.371195:0:2403:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2403:1862022114154496:192.168.202.110@tcp:400:kworker.0
00000100:00100000:1.0:1775764857.371204:0:2403:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2403:1862022114154752:192.168.202.110@tcp:400:kworker.0
00000100:00100000:1.0:1775764857.371213:0:2403:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2403:1862022114155008:192.168.202.110@tcp:400:kworker.0
00010000:00010000:1.0:1775764860.177679:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775764862.257537:0:2395:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.110@tcp (source: 12345-192.168.202.110@tcp)
00000100:00100000:1.0:1775764862.257615:0:39305:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1862022124537088
00000100:00100000:1.0:1775764862.257626:0:39305:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1862022124537088
00000100:00100000:1.0:1775764862.257680:0:39305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.110@tcp, seq: 6417
00000100:00100000:1.0:1775764862.257699:0:39305:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9f14ab4fca80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48+5:6485:x1862022124537088:12345-192.168.202.110@tcp:104:
00010000:00010000:1.0:1775764862.257719:0:39305:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast  ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f1498179400/0x23cab7dad48e60d3 lrc: 2/0,0 mode: PR/PR res: [0x200000405:0x7b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x420000000000 nid: local remote: 0x92930478723306e expref: -99 pid: 72268 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0
00010000:00010000:1.0:1775764862.257796:0:39305:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool
00000100:00100000:1.0:1775764862.257803:0:39305:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9f14ab4fca80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48+5:6485:x1862022124537088:12345-192.168.202.110@tcp:104: Request processed in 107us (288us total) trans 0 rc 0/0
00000100:00100000:1.0:1775764862.257811:0:39305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.110@tcp, seq: 6417
00010000:00010000:1.0:1775764862.388747:0:46896:0:(ldlm_lockd.c:2794:ldlm_bl_get_work()) Got 1 locks of 0 total in blp.  (0 blwis in pool)
00010000:00010000:1.0:1775764862.388756:0:46896:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f1498178400/0x23cab7dad48e60ef lrc: 2/0,0 mode: PR/PR res: [0x200000405:0x7a:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x400000000 nid: local remote: 0x9293047872330d0 expref: -99 pid: 72275 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0
00010000:00010000:1.0:1775764862.388764:0:46896:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f1498178400 already unused, calling callback (ffffffffc1a0c2e0)
00010000:00010000:1.0:1775764862.388769:0:46896:0:(ldlm_request.c:1724:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f1498178400/0x23cab7dad48e60ef lrc: 2/0,0 mode: PR/PR res: [0x200000405:0x7a:0x0].0x0 bits 0x1b/0x1b rrc: 3 type: IBT gid 0 flags: 0x400000000 nid: local remote: 0x9293047872330d0 expref: -99 pid: 72275 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0
00010000:00010000:1.0:1775764862.388778:0:46896:0:(ldlm_request.c:1734:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f1498178400/0x23cab7dad48e60ef lrc: 2/0,0 mode: PR/PR res: [0x200000405:0x7a:0x0].0x0 bits 0x1b/0x1b rrc: 3 type: IBT gid 0 flags: 0x400000000 nid: local remote: 0x9293047872330d0 expref: -99 pid: 72275 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0
00010000:00010000:1.0:1775764862.388786:0:46896:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f1498178400/0x23cab7dad48e60ef lrc: 3/0,0 mode: PR/PR res: [0x200000405:0x7a:0x0].0x0 bits 0x1b/0x1b rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x9293047872330d0 expref: -99 pid: 72275 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0
00010000:00010000:1.0:1775764862.388804:0:46896:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f149153e400/0x23cab7dad48e60b7 lrc: 2/0,0 mode: PR/PR res: [0x200000405:0x7a:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x929304787233021 expref: -99 pid: 72267 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0
00010000:00010000:1.0:1775764862.388840:0:46896:0:(ldlm_request.c:1438:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f1498178400/0x23cab7dad48e60ef lrc: 2/0,0 mode: --/PR res: [0x200000405:0x7a:0x0].0x0 bits 0x1b/0x1b rrc: 3 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x9293047872330d0 expref: -99 pid: 72275 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0
00010000:00010000:1.0:1775764862.388846:0:46896:0:(ldlm_request.c:1490:ldlm_cancel_pack()) 1 locks packed
00010000:00010000:1.0:1775764862.388871:0:46896:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f1498178400/0x23cab7dad48e60ef lrc: 1/0,0 mode: --/PR res: [0x200000405:0x7a:0x0].0x0 bits 0x1b/0x1b rrc: 3 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x9293047872330d0 expref: -99 pid: 72275 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0
00010000:00010000:1.0:1775764862.388878:0:46896:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f1498178400/0x23cab7dad48e60ef lrc: 0/0,0 mode: --/PR res: [0x200000405:0x7a:0x0].0x0 bits 0x1b/0x1b rrc: 3 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x9293047872330d0 expref: -99 pid: 72275 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0
00010000:00010000:1.0:1775764862.388893:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00010000:00010000:1.0:1775764862.388895:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00010000:00010000:1.0:1775764862.388897:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775764862.388917:0:2404:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2404:1862022114157952:192.168.202.110@tcp:103:ldlm_bl.0
00000080:00200000:1.0:1775764862.600521:0:72277:0:(file.c:6121:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9f1491deba88),name=/
00000002:00010000:1.0:1775764862.600573:0:72277:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0
00010000:00010000:1.0:1775764862.601364:0:72277:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f14ab4c4a00/0x23cab7dad48e6120 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 72277 initiator: MDT0 lru_score: 0 lru_type: 0
00010000:00010000:1.0:1775764862.601380:0:72277:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f14ab4c4a00/0x23cab7dad48e6120 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 72277 initiator: MDT0 lru_score: 0 lru_type: 0
00010000:00010000:1.0:1775764862.601393:0:72277:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f14ab4c4a00/0x23cab7dad48e6120 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 72277 initiator: MDT0 lru_score: 0 lru_type: 0
00000100:00100000:1.0:1775764862.601422:0:72277:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ffb80 pname:cluuid:pid:xid:nid:opc:job lfs:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:72277:1862022114160384:192.168.202.110@tcp:101:lfs.0
00000100:00100000:1.0:1775764862.601641:0:72277:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14813709c0 going to sleep for 50 seconds
00000080:00200000:1.0:1775764863.906250:0:72291:0:(file.c:6121:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9f1491deba88),name=/
00000002:00010000:1.0:1775764863.906280:0:72291:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0
00010000:00010000:1.0:1775764863.906299:0:72291:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f14ab4c4a00/0x23cab7dad48e6120 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x929304787233140 expref: -99 pid: 72277 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 0
00010000:00010000:1.0:1775764863.906315:0:72291:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f14ab4c4a00/0x23cab7dad48e6120 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x929304787233140 expref: -99 pid: 72277 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 0
00000080:00200000:1.0:1775764863.906324:0:72291:0:(namei.c:953:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1
00000080:00010000:1.0:1775764863.906332:0:72291:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9f1491deba88)
00000080:00010000:1.0:1775764863.906335:0:72291:0:(llite_internal.h:1984:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9f1491deba88) for lock 0x23cab7dad48e6120
00000080:00010000:1.0:1775764863.906418:0:72291:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x23cab7dad48e6120 from it ffffb06c014a3bf8
00010000:00010000:1.0:1775764863.906422:0:72291:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f14ab4c4a00/0x23cab7dad48e6120 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x929304787233140 expref: -99 pid: 72277 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 0
00010000:00010000:1.0:1775764863.906437:0:72291:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f14ab4c4a00/0x23cab7dad48e6120 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x929304787233140 expref: -99 pid: 72277 timeout: 0 lvb_type: 0 lru_score: 7 lru_type: 0
00000080:00200000:1.0:1775764863.906453:0:72291:0:(file.c:6593:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9f1491deba88), inode mode 41ed mask 1
00000080:00200000:1.0:1775764863.906460:0:72291:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=f207.sanity-flr, flags=265
00000080:00200000:1.0:1775764863.906465:0:72291:0:(file.c:6593:ll_inode_permission()) VFS Op:inode=[0x200000405:0x7b:0x0](ffff9f14a1111148), inode mode 81a4 mask 46
00000080:00200008:1.0:1775764863.906472:0:72291:0:(file.c:1057:ll_file_open()) START file f207.sanity-flr:[0x200000405:0x7b:0x0](ffff9f14a1111148), flags 1154402
00000100:00100000:1.0:1775764863.932186:0:44145:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1862022124541184
00000100:00100000:1.0:1775764863.932197:0:44145:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1862022124541184
00000100:00100000:1.0:1775764863.932303:0:44145:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.110@tcp, seq: 6418
00000100:00100000:1.0:1775764863.932313:0:44145:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9f14a04d2d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48+5:6485:x1862022124541184:12345-192.168.202.110@tcp:104:
00010000:00010000:1.0:1775764863.932418:0:44145:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast  ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f149817ea00/0x23cab7dad48e6143 lrc: 2/0,0 mode: PR/PR res: [0x200000405:0x7d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x420000000000 nid: local remote: 0x9293047872331e8 expref: -99 pid: 72291 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0
00010000:00010000:1.0:1775764863.932517:0:44145:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool
00000100:00100000:1.0:1775764863.932525:0:44145:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9f14a04d2d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48+5:6485:x1862022124541184:12345-192.168.202.110@tcp:104: Request processed in 213us (646us total) trans 0 rc 0/0
00000100:00100000:1.0:1775764863.932533:0:44145:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.110@tcp, seq: 6418
00000100:00100000:1.0:1775764864.014008:0:2404:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2404:1862022114166400:192.168.202.110@tcp:101:lfs.0
00000100:00100000:1.0:1775764864.014103:0:2404:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1]
00000100:00100000:1.0:1775764864.014110:0:2404:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2404:1862022114166272:192.168.202.110@tcp:101:lfs.0
00000100:00100000:1.0:1775764864.014163:0:2403:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2403:1862022114166144:192.168.202.110@tcp:103:ldlm_bl.0
00010000:00010000:1.0:1775764864.017279:0:2404:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-OST0001-osc-ffff9f14902b3800 lock: 00000000278d5f64/0x23cab7dad48e616d lrc: 6/2,0 mode: --/PR res: [0x2c0000401:0xb9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 72291 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0
00010000:00010000:1.0:1775764864.017314:0:2404:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-OST0001-osc-ffff9f14902b3800 lock: 00000000278d5f64/0x23cab7dad48e616d lrc: 6/2,0 mode: --/PR res: [0x2c0000401:0xb9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x0 nid: local remote: 0x929304787233251 expref: -99 pid: 72291 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0
00010000:00010000:1.0:1775764864.017330:0:2404:0:(ldlm_resource.c:1714:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-OST0001-osc-ffff9f14902b3800 lock: 00000000278d5f64/0x23cab7dad48e616d lrc: 6/2,0 mode: PR/PR res: [0x2c0000401:0xb9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10000000000000 nid: local remote: 0x929304787233251 expref: -99 pid: 72291 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0
00010000:00010000:1.0:1775764864.017344:0:2404:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-OST0001-osc-ffff9f14902b3800 lock: 00000000278d5f64/0x23cab7dad48e616d lrc: 6/2,0 mode: PR/PR res: [0x2c0000401:0xb9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x0 nid: local remote: 0x929304787233251 expref: -99 pid: 72291 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0
00000008:00010000:1.0:1775764864.017361:0:2404:0:(osc_lock.c:187:osc_lock_lvb_update()) ### acquired size 1048576, setting rss=1048576; kms=1048576, end=18446744073709551615 ns: lustre-OST0001-osc-ffff9f14902b3800 lock: 00000000278d5f64/0x23cab7dad48e616d lrc: 5/2,0 mode: PR/PR res: [0x2c0000401:0xb9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10020000000000 nid: local remote: 0x929304787233251 expref: -99 pid: 72291 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0
00000020:00200000:1.0:1775764864.017374:0:2404:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining.
00010000:00010000:1.0:1775764864.017379:0:2404:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-OST0001-osc-ffff9f14902b3800 lock: 00000000278d5f64/0x23cab7dad48e616d lrc: 6/2,0 mode: PR/PR res: [0x2c0000401:0xb9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x810020000000000 nid: local remote: 0x929304787233251 expref: -99 pid: 72291 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0
00010000:00010000:1.0:1775764864.017386:0:2404:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-OST0001-osc-ffff9f14902b3800 lock: 00000000278d5f64/0x23cab7dad48e616d lrc: 5/1,0 mode: PR/PR res: [0x2c0000401:0xb9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x810020000000000 nid: local remote: 0x929304787233251 expref: -99 pid: 72291 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0
00010000:00010000:1.0:1775764864.017393:0:2404:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-OST0001-osc-ffff9f14902b3800 lock: 00000000278d5f64/0x23cab7dad48e616d lrc: 5/1,0 mode: PR/PR res: [0x2c0000401:0xb9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x810020000000000 nid: local remote: 0x929304787233251 expref: -99 pid: 72291 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0
00010000:00010000:1.0:1775764864.017401:0:2404:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-OST0001-osc-ffff9f14902b3800 lock: 00000000278d5f64/0x23cab7dad48e616d lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0xb9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800020000000000 nid: local remote: 0x929304787233251 expref: -99 pid: 72291 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0
00000100:00100000:1.0:1775764864.017427:0:2404:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2404:1862022114166272:192.168.202.110@tcp:101:lfs.0
00010000:00010000:1.0:1775764864.017447:0:2404:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-OST0000-osc-ffff9f14902b3800 lock: 000000003e4a7c12/0x23cab7dad48e6174 lrc: 6/2,0 mode: --/PR res: [0x280000401:0xb8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 72291 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0
00010000:00010000:1.0:1775764864.017458:0:2404:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-OST0000-osc-ffff9f14902b3800 lock: 000000003e4a7c12/0x23cab7dad48e6174 lrc: 6/2,0 mode: --/PR res: [0x280000401:0xb8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x0 nid: local remote: 0x92930478723324a expref: -99 pid: 72291 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0
00010000:00010000:1.0:1775764864.017466:0:2404:0:(ldlm_resource.c:1714:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-OST0000-osc-ffff9f14902b3800 lock: 000000003e4a7c12/0x23cab7dad48e6174 lrc: 6/2,0 mode: PR/PR res: [0x280000401:0xb8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10000000000000 nid: local remote: 0x92930478723324a expref: -99 pid: 72291 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0
00010000:00010000:1.0:1775764864.017473:0:2404:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-OST0000-osc-ffff9f14902b3800 lock: 000000003e4a7c12/0x23cab7dad48e6174 lrc: 6/2,0 mode: PR/PR res: [0x280000401:0xb8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x0 nid: local remote: 0x92930478723324a expref: -99 pid: 72291 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0
00000008:00010000:1.0:1775764864.017495:0:2404:0:(osc_lock.c:187:osc_lock_lvb_update()) ### acquired size 0, setting rss=0; kms=0, end=18446744073709551615 ns: lustre-OST0000-osc-ffff9f14902b3800 lock: 000000003e4a7c12/0x23cab7dad48e6174 lrc: 5/2,0 mode: PR/PR res: [0x280000401:0xb8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10020000000000 nid: local remote: 0x92930478723324a expref: -99 pid: 72291 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0
00000020:00200000:1.0:1775764864.017504:0:2404:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining.
00010000:00010000:1.0:1775764864.017517:0:2404:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-OST0000-osc-ffff9f14902b3800 lock: 000000003e4a7c12/0x23cab7dad48e6174 lrc: 6/2,0 mode: PR/PR res: [0x280000401:0xb8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x810020000000000 nid: local remote: 0x92930478723324a expref: -99 pid: 72291 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0
00010000:00010000:1.0:1775764864.017524:0:2404:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-OST0000-osc-ffff9f14902b3800 lock: 000000003e4a7c12/0x23cab7dad48e6174 lrc: 5/1,0 mode: PR/PR res: [0x280000401:0xb8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x810020000000000 nid: local remote: 0x92930478723324a expref: -99 pid: 72291 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0
00010000:00010000:1.0:1775764864.017531:0:2404:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-OST0000-osc-ffff9f14902b3800 lock: 000000003e4a7c12/0x23cab7dad48e6174 lrc: 5/1,0 mode: PR/PR res: [0x280000401:0xb8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x810020000000000 nid: local remote: 0x92930478723324a expref: -99 pid: 72291 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0
00010000:00010000:1.0:1775764864.017539:0:2404:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-OST0000-osc-ffff9f14902b3800 lock: 000000003e4a7c12/0x23cab7dad48e6174 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0xb8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800020000000000 nid: local remote: 0x92930478723324a expref: -99 pid: 72291 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0
00000100:00100000:1.0:1775764864.017549:0:2404:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2404:1862022114166400:192.168.202.110@tcp:101:lfs.0
00000080:00200008:1.0:1775764864.060042:0:72291:0:(file.c:5170:ll_file_seek()) START file f207.sanity-flr:[0x200000405:0x7b:0x0], offset: 0, type: (null)
00000080:00200000:1.0:1775764864.060092:0:72291:0:(vvp_io.c:1817:vvp_io_init()) [0x200000405:0x7b:0x0] ignore/verify layout 0/1, layout version 0 restore needed 0
00020000:00200000:1.0:1775764864.060105:0:72291:0:(lov_io.c:458:lov_io_mirror_init()) [0x200000405:0x7b:0x0]: flr state: 2, move mirror from 0 to 0, have retried: 0, mirror count: 2
00020000:00200000:1.0:1775764864.060108:0:72291:0:(lov_io.c:484:lov_io_mirror_init()) use non-delayed RPC state for this IO
00000001:00010000:1.0:1775764864.060111:0:72291:0:(glimpse.c:73:cl_glimpse_lock()) Glimpsing inode [0x200000405:0x7b:0x0]
00000008:00010000:1.0:1775764864.060146:0:72291:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: a
00000008:00010000:1.0:1775764864.060150:0:72291:0:(osc_lock.c:1310:osc_lock_init()) ### lock ffff9f14899bbc30, osc lock ffff9f148759ad10, flags 0x400000400001002
00000008:00010000:1.0:1775764864.060169:0:72291:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: a
00000008:00010000:1.0:1775764864.060170:0:72291:0:(osc_lock.c:1310:osc_lock_init()) ### lock ffff9f14899bbc70, osc lock ffff9f148759a000, flags 0x400000400001002
00010000:00010000:1.0:1775764864.060195:0:72291:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-OST0001-osc-ffff9f14902b3800 lock: 00000000278d5f64/0x23cab7dad48e616d lrc: 2/1,0 mode: PR/PR res: [0x2c0000401:0xb9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800020000000000 nid: local remote: 0x929304787233251 expref: -99 pid: 72291 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0
00010000:00010000:1.0:1775764864.060212:0:72291:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 18446744073709551615) ns: lustre-OST0001-osc-ffff9f14902b3800 lock: 00000000278d5f64/0x23cab7dad48e616d lrc: 2/1,0 mode: PR/PR res: [0x2c0000401:0xb9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800020000000000 nid: local remote: 0x929304787233251 expref: -99 pid: 72291 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0
00000020:00200000:1.0:1775764864.060224:0:72291:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining.
00010000:00010000:1.0:1775764864.060227:0:72291:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-OST0001-osc-ffff9f14902b3800 lock: 00000000278d5f64/0x23cab7dad48e616d lrc: 5/1,0 mode: PR/PR res: [0x2c0000401:0xb9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x810020000000000 nid: local remote: 0x929304787233251 expref: -99 pid: 72291 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0
00010000:00010000:1.0:1775764864.060233:0:72291:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-OST0001-osc-ffff9f14902b3800 lock: 00000000278d5f64/0x23cab7dad48e616d lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0xb9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800020000000000 nid: local remote: 0x929304787233251 expref: -99 pid: 72291 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0
00010000:00010000:1.0:1775764864.060253:0:72291:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-OST0000-osc-ffff9f14902b3800 lock: 000000003e4a7c12/0x23cab7dad48e6174 lrc: 2/1,0 mode: PR/PR res: [0x280000401:0xb8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800020000000000 nid: local remote: 0x92930478723324a expref: -99 pid: 72291 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0
00010000:00010000:1.0:1775764864.060258:0:72291:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 18446744073709551615) ns: lustre-OST0000-osc-ffff9f14902b3800 lock: 000000003e4a7c12/0x23cab7dad48e6174 lrc: 2/1,0 mode: PR/PR res: [0x280000401:0xb8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800020000000000 nid: local remote: 0x92930478723324a expref: -99 pid: 72291 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0
00000020:00200000:1.0:1775764864.060262:0:72291:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining.
00010000:00010000:1.0:1775764864.060263:0:72291:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-OST0000-osc-ffff9f14902b3800 lock: 000000003e4a7c12/0x23cab7dad48e6174 lrc: 5/1,0 mode: PR/PR res: [0x280000401:0xb8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x810020000000000 nid: local remote: 0x92930478723324a expref: -99 pid: 72291 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0
00010000:00010000:1.0:1775764864.060267:0:72291:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-OST0000-osc-ffff9f14902b3800 lock: 000000003e4a7c12/0x23cab7dad48e6174 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0xb8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800020000000000 nid: local remote: 0x92930478723324a expref: -99 pid: 72291 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0
00000020:00200000:1.0:1775764864.060272:0:72291:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining.
00000080:00200000:1.0:1775764864.060280:0:72291:0:(file.c:1742:ll_merge_attr_nolock()) [0x200000405:0x7b:0x0] updating i_size 1048576 i_blocks 2048
00000020:00010000:1.0:1775764864.060284:0:72291:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff9f1486116910 (ffff9f1487c50ab8/0) at cl_lock_release():245
00000020:00010000:1.0:1775764864.060286:0:72291:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9f1486116910 (ffff9f1487c50ab8/0) at cl_lock_cancel():142
00000020:00010000:1.0:1775764864.060287:0:72291:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9f14899bbc30 (ffff9f1487c50098/1) at cl_lock_cancel():142
00000020:00010000:1.0:1775764864.060290:0:72291:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9f14899bbc70 (ffff9f1487c50368/1) at cl_lock_cancel():142
00000020:00010000:1.0:1775764864.060291:0:72291:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9f1486116910 (ffff9f1487c50ab8/0) at cl_lock_fini():72
00000020:00010000:1.0:1775764864.060293:0:72291:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9f14899bbc30 (ffff9f1487c50ab8/1) at cl_lock_fini():72
00000020:00010000:1.0:1775764864.060298:0:72291:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9f14899bbc70 (ffff9f1487c50ab8/1) at cl_lock_fini():72
00000080:00200000:1.0:1775764864.060325:0:72291:0:(vvp_io.c:288:vvp_io_fini()) [0x200000405:0x7b:0x0] ignore/verify layout 0/1, layout version 3 need write layout 0, restore needed 0, invalidate_lock 0
00000080:00200008:1.0:1775764864.060337:0:72291:0:(file.c:5201:ll_file_seek()) COMPLETED file f207.sanity-flr:[0x200000405:0x7b:0x0], offset: 0, type: (null), rc = 1048576
00000080:00200008:1.0:1775764864.060342:0:72291:0:(file.c:5170:ll_file_seek()) START file f207.sanity-flr:[0x200000405:0x7b:0x0], offset: 0, type: SEEK_HOLE
00000080:00200000:1.0:1775764864.060354:0:72291:0:(vvp_io.c:1817:vvp_io_init()) [0x200000405:0x7b:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0
00020000:00200000:1.0:1775764864.060358:0:72291:0:(lov_io.c:458:lov_io_mirror_init()) [0x200000405:0x7b:0x0]: flr state: 2, move mirror from 0 to 0, have retried: 0, mirror count: 2
00020000:00200000:1.0:1775764864.060360:0:72291:0:(lov_io.c:484:lov_io_mirror_init()) use delayed RPC state for this IO
00020000:00200000:1.0:1775764864.060364:0:72291:0:(lov_io.c:860:lov_io_iter_init()) component[0] flags 0x10
00000080:00200000:1.0:1775764864.060393:0:72291:0:(file.c:1742:ll_merge_attr_nolock()) [0x200000405:0x7b:0x0] updating i_size 1048576 i_blocks 2048
00000080:00200000:1.0:1775764864.060411:0:72291:0:(vvp_io.c:288:vvp_io_fini()) [0x200000405:0x7b:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0
00000080:00200000:1.0:1775764864.060423:0:72291:0:(file.c:1815:ll_io_set_mirror()) f207.sanity-flr: desiginated mirror: 0
00000080:00200000:1.0:1775764864.060425:0:72291:0:(vvp_io.c:1817:vvp_io_init()) [0x200000405:0x7b:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0
00020000:00200000:1.0:1775764864.060428:0:72291:0:(lov_io.c:458:lov_io_mirror_init()) [0x200000405:0x7b:0x0]: flr state: 2, move mirror from 0 to 0, have retried: 0, mirror count: 2
00020000:00200000:1.0:1775764864.060430:0:72291:0:(lov_io.c:484:lov_io_mirror_init()) use delayed RPC state for this IO
00020000:00200000:1.0:1775764864.060431:0:72291:0:(lov_io.c:860:lov_io_iter_init()) component[0] flags 0x10
00000080:00200000:1.0:1775764864.060441:0:72291:0:(vvp_io.c:198:vvp_io_one_lock_index()) lock: 0 [0, 18446744073709551615]
00000008:00010000:1.0:1775764864.060446:0:72291:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 8
00000008:00010000:1.0:1775764864.060447:0:72291:0:(osc_lock.c:1310:osc_lock_init()) ### lock ffff9f14899bbc30, osc lock ffff9f148759a000, flags 0x0
00000008:00010000:1.0:1775764864.060449:0:72291:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 8
00000008:00010000:1.0:1775764864.060449:0:72291:0:(osc_lock.c:1310:osc_lock_init()) ### lock ffff9f14899bbc70, osc lock ffff9f148759ad10, flags 0x0
00010000:00010000:1.0:1775764864.060456:0:72291:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-OST0001-osc-ffff9f14902b3800 lock: 00000000278d5f64/0x23cab7dad48e616d lrc: 2/1,0 mode: PR/PR res: [0x2c0000401:0xb9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800020000000000 nid: local remote: 0x929304787233251 expref: -99 pid: 72291 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0
00010000:00010000:1.0:1775764864.060463:0:72291:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 18446744073709551615) ns: lustre-OST0001-osc-ffff9f14902b3800 lock: 00000000278d5f64/0x23cab7dad48e616d lrc: 2/1,0 mode: PR/PR res: [0x2c0000401:0xb9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800020000000000 nid: local remote: 0x929304787233251 expref: -99 pid: 72291 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0
00010000:00010000:1.0:1775764864.060468:0:72291:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-OST0001-osc-ffff9f14902b3800 lock: 00000000278d5f64/0x23cab7dad48e616d lrc: 6/2,0 mode: PR/PR res: [0x2c0000401:0xb9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x810020000000000 nid: local remote: 0x929304787233251 expref: -99 pid: 72291 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0
00010000:00010000:1.0:1775764864.060473:0:72291:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-OST0001-osc-ffff9f14902b3800 lock: 00000000278d5f64/0x23cab7dad48e616d lrc: 6/2,0 mode: PR/PR res: [0x2c0000401:0xb9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x810020000000000 nid: local remote: 0x929304787233251 expref: -99 pid: 72291 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0
00010000:00010000:1.0:1775764864.060481:0:72291:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-OST0001-osc-ffff9f14902b3800 lock: 00000000278d5f64/0x23cab7dad48e616d lrc: 5/1,0 mode: PR/PR res: [0x2c0000401:0xb9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x810020000000000 nid: local remote: 0x929304787233251 expref: -99 pid: 72291 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0
00010000:00010000:1.0:1775764864.060486:0:72291:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-OST0000-osc-ffff9f14902b3800 lock: 000000003e4a7c12/0x23cab7dad48e6174 lrc: 2/1,0 mode: PR/PR res: [0x280000401:0xb8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800020000000000 nid: local remote: 0x92930478723324a expref: -99 pid: 72291 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0
00010000:00010000:1.0:1775764864.060490:0:72291:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 18446744073709551615) ns: lustre-OST0000-osc-ffff9f14902b3800 lock: 000000003e4a7c12/0x23cab7dad48e6174 lrc: 2/1,0 mode: PR/PR res: [0x280000401:0xb8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800020000000000 nid: local remote: 0x92930478723324a expref: -99 pid: 72291 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0
00010000:00010000:1.0:1775764864.060495:0:72291:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-OST0000-osc-ffff9f14902b3800 lock: 000000003e4a7c12/0x23cab7dad48e6174 lrc: 6/2,0 mode: PR/PR res: [0x280000401:0xb8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x810020000000000 nid: local remote: 0x92930478723324a expref: -99 pid: 72291 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0
00010000:00010000:1.0:1775764864.060499:0:72291:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-OST0000-osc-ffff9f14902b3800 lock: 000000003e4a7c12/0x23cab7dad48e6174 lrc: 6/2,0 mode: PR/PR res: [0x280000401:0xb8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x810020000000000 nid: local remote: 0x92930478723324a expref: -99 pid: 72291 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0
00010000:00010000:1.0:1775764864.060503:0:72291:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-OST0000-osc-ffff9f14902b3800 lock: 000000003e4a7c12/0x23cab7dad48e6174 lrc: 5/1,0 mode: PR/PR res: [0x280000401:0xb8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x810020000000000 nid: local remote: 0x92930478723324a expref: -99 pid: 72291 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0
00000080:00200000:1.0:1775764864.060508:0:72291:0:(file.c:1742:ll_merge_attr_nolock()) [0x200000405:0x7b:0x0] updating i_size 1048576 i_blocks 2048
00000100:00100000:1.0:1775764864.065917:0:2403:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2403:1862022114166528:192.168.202.110@tcp:23:lfs.0
00000080:00200000:1.0:1775764864.097471:0:72291:0:(file.c:2173:ll_file_io_generic()) Range unlock [0, 16383]
00000080:00200000:1.0:1775764864.097531:0:72291:0:(vvp_io.c:288:vvp_io_fini()) [0x200000405:0x7b:0x0] ignore/verify layout 0/0, layout version 3 need write layout 0, restore needed 0, invalidate_lock 0
00000080:00200000:1.0:1775764864.097538:0:72291:0:(file.c:2206:ll_file_io_generic()) f207.sanity-flr: 1 io complete with rc: 0, result: 1048576, restart: 0
00000020:00200000:1.0:1775764864.097544:0:72291:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining.
00000080:00200000:1.0:1775764864.097576:0:72291:0:(file.c:2292:ll_file_io_generic()) iot: 1, result: 1048576
00000080:00200008:1.0:1775764864.097616:0:72291:0:(file.c:2612:do_file_write_iter()) START file .:VOLATILE:0000:2E51FE8E:fd=03:[0x200000405:0x7d:0x0], ppos: 0, count: 1048576
00000080:00200000:1.0:1775764864.097643:0:72291:0:(file.c:2058:ll_file_io_generic()) .:VOLATILE:0000:2E51FE8E:fd=03: write ppos: 0, bytes: 1048576
00000080:00200000:1.0:1775764864.097653:0:72291:0:(file.c:1815:ll_io_set_mirror()) .:VOLATILE:0000:2E51FE8E:fd=03: desiginated mirror: 0
00000020:00200000:1.0:1775764864.097661:0:72291:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e99680f2[0x110, 2, [0x200000405:0x7d:0x0] hash]
00000020:00200000:1.0:1775764864.097663:0:72291:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [0, 1048576) 2048 0
00000080:00200000:1.0:1775764864.097679:0:72291:0:(vvp_io.c:1817:vvp_io_init()) [0x200000405:0x7d:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0
00000080:00200000:1.0:1775764864.097698:0:72291:0:(file.c:2136:ll_file_io_generic()) Range lock [0, 255]
00020000:00200000:1.0:1775764864.097706:0:72291:0:(lov_io.c:1052:lov_io_rw_iter_init()) stripe: 0 chunk: [0, 1048576) 1048576, 1048576
00020000:00200000:1.0:1775764864.097710:0:72291:0:(lov_io.c:860:lov_io_iter_init()) component[0] flags 0x10
00000080:00200000:1.0:1775764864.097742:0:72291:0:(vvp_io.c:198:vvp_io_one_lock_index()) lock: 1 [0, 255]
00000008:00010000:1.0:1775764864.097759:0:72291:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 11
00000008:00010000:1.0:1775764864.097764:0:72291:0:(osc_lock.c:1310:osc_lock_init()) ### lock ffff9f14945e0330, osc lock ffff9f148759bb50, flags 0x40000
00000080:00200000:1.0:1775764864.097775:0:72291:0:(vvp_io.c:1286:vvp_io_write_start()) .:VOLATILE:0000:2E51FE8E:fd=03: write [0, 1048576) append 0 append lockpos 0
00000002:00010000:1.0:1775764864.097813:0:72291:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: ,[0x0:0x0:0x0]) in obj [0x200000405:0x7d:0x0], intent: getxattr flags 0
00010000:00010000:1.0:1775764864.097889:0:72291:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f14a04c4000/0x23cab7dad48e617b lrc: 3/1,0 mode: --/PR res: [0x200000405:0x7d:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 72291 initiator: MDT0 lru_score: 0 lru_type: 0
00010000:00010000:1.0:1775764864.097899:0:72291:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f14a04c4000/0x23cab7dad48e617b lrc: 3/1,0 mode: --/PR res: [0x200000405:0x7d:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x0 pid: 72291 initiator: MDT0 lru_score: 0 lru_type: 0
00010000:00010000:1.0:1775764864.097910:0:72291:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f14a04c4000/0x23cab7dad48e617b lrc: 3/1,0 mode: --/PR res: [0x200000405:0x7d:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x0 pid: 72291 initiator: MDT0 lru_score: 0 lru_type: 0
00000100:00100000:1.0:1775764864.097934:0:72291:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff800 pname:cluuid:pid:xid:nid:opc:job lfs:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:72291:1862022114166784:192.168.202.110@tcp:101:lfs.0
00000100:00100000:1.0:1775764864.098053:0:72291:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14899bb540 going to sleep for 50 seconds
00000100:00100000:1.0:1775764864.121309:0:72291:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff800 pname:cluuid:pid:xid:nid:opc:job lfs:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:72291:1862022114166784:192.168.202.110@tcp:101:lfs.0
00010000:00010000:1.0:1775764864.121339:0:72291:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f14a04c4000/0x23cab7dad48e617b lrc: 4/1,0 mode: --/PR res: [0x200000405:0x7d:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x929304787233266 expref: -99 pid: 72291 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0
00010000:00010000:1.0:1775764864.121360:0:72291:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f14a04c4000/0x23cab7dad48e617b lrc: 4/1,0 mode: PR/PR res: [0x200000405:0x7d:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x929304787233266 expref: -99 pid: 72291 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0
00010000:00010000:1.0:1775764864.121370:0:72291:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f14a04c4000/0x23cab7dad48e617b lrc: 4/1,0 mode: PR/PR res: [0x200000405:0x7d:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x929304787233266 expref: -99 pid: 72291 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0
00000002:00100000:1.0:1775764864.121385:0:72291:0:(mdc_locks.c:833:mdc_finish_enqueue()) @@@ op=80 disposition=0, status=0  req@ffff9f14ab4ff800 x1862022114166784/t0(0) o101->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 568/1176 e 0 to 0 dl 1775764919 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'lfs.0' uid:0 gid:0 projid:0
00000002:00010000:1.0:1775764864.121396:0:72291:0:(mdc_locks.c:1357:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f14a04c4000/0x23cab7dad48e617b lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x7d:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x929304787233266 expref: -99 pid: 72291 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0
00010000:00010000:1.0:1775764864.121409:0:72291:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9f1494571c00 type 13 mode 4 res 8589935621/125 (0 0)
00000080:00200000:1.0:1775764864.121414:0:72291:0:(namei.c:953:ll_intent_lock()) intent lock 128 on i1 [0x200000405:0x7d:0x0] suppgids 0 -1: rc 0
00000080:00010000:1.0:1775764864.121437:0:72291:0:(llite_internal.h:1984:ll_set_lock_data()) setting l_data to inode [0x200000405:0x7d:0x0](ffff9f1491d35b88) for lock 0x23cab7dad48e617b
00000080:00010000:1.0:1775764864.121445:0:72291:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x23cab7dad48e617b from it ffffb06c014a38f8
00010000:00010000:1.0:1775764864.121449:0:72291:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f14a04c4000/0x23cab7dad48e617b lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x7d:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x929304787233266 expref: -99 pid: 72291 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0
00010000:00010000:1.0:1775764864.121458:0:72291:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f14a04c4000/0x23cab7dad48e617b lrc: 2/0,0 mode: PR/PR res: [0x200000405:0x7d:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x929304787233266 expref: -99 pid: 72291 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0
00000080:00200000:1.0:1775764864.121511:0:72291:0:(rw26.c:395:ll_direct_IO()) VFS Op:inode=[0x200000405:0x7d:0x0](ffff9f1491d35b88), size=1048576 (max 536870912), offset=0=0x0, pages 256 (max 131072), lockless, parallel
00000020:00200000:1.0:1775764864.121872:0:72291:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 256 items remaining.
00000080:00200000:1.0:1775764864.121876:0:72291:0:(rw26.c:531:ll_direct_IO()) result 0 tot_bytes 1048576 count 1048576 file_offset 1048576
00000080:00200000:1.0:1775764864.121906:0:72291:0:(file.c:1742:ll_merge_attr_nolock()) [0x200000405:0x7d:0x0] updating i_size 0 i_blocks 0
00000020:00010000:1.0:1775764864.121912:0:72291:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff9f14a6766218 (ffff9f1487c50248/0) at cl_lock_release():245
00000020:00010000:1.0:1775764864.121916:0:72291:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9f14a6766218 (ffff9f1487c50248/0) at cl_lock_cancel():142
00000020:00010000:1.0:1775764864.121920:0:72291:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9f14945e0330 (ffff9f1487c50bd8/1) at cl_lock_cancel():142
00000020:00010000:1.0:1775764864.121924:0:72291:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9f14a6766218 (ffff9f1487c50248/0) at cl_lock_fini():72
00000020:00010000:1.0:1775764864.121926:0:72291:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9f14945e0330 (ffff9f1487c50248/1) at cl_lock_fini():72
00000080:00200000:1.0:1775764864.121935:0:72291:0:(vvp_io.c:517:vvp_io_advance()) advancing 1048576 bytes
00000020:00200000:1.0:1775764864.121942:0:72291:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining.
00000008:00010000:1.0:1775764864.121990:0:70315:0:(osc_cache.c:2012:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff9f14a04c9240
00000008:00100000:1.0:1775764864.125387:0:70315:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff9f14ab4ff800 - object 0x280000401:186 offset 0<>1048576
00000100:00100000:1.0:1775764864.141628:0:39305:0:(service.c:153:ptlrpc_grow_req_bufs()) ldlm_cbd: allocate 1 new 8192-byte reqbufs (0/2 left), rc = 0
00000100:00100000:1.0:1775764864.141651:0:39305:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1862022124541568
00000100:00100000:1.0:1775764864.141660:0:39305:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1862022124541568
00000100:00100000:1.0:1775764864.141741:0:39305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.110@tcp, seq: 6421
00000100:00100000:1.0:1775764864.141752:0:39305:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9f14a662f050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:LOV_OSC_UUID+4:23978:x1862022124541568:12345-192.168.202.110@tcp:104:
00010000:00010000:1.0:1775764864.141770:0:39305:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast  ns: lustre-OST0000-osc-ffff9f14902b3800 lock: 0000000039155643/0x23cab7dad48e6158 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0xba:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800420000000000 nid: local remote: 0x929304787233219 expref: -99 pid: 72291 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0
00010000:00010000:1.0:1775764864.141857:0:39305:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool
00000100:00100000:1.0:1775764864.141865:0:39305:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9f14a662f050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:LOV_OSC_UUID+4:23978:x1862022124541568:12345-192.168.202.110@tcp:104: Request processed in 115us (1077us total) trans 0 rc 0/0
00000100:00100000:1.0:1775764864.141872:0:39305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.110@tcp, seq: 6421
00000100:00100000:1.0:1775764864.183141:0:2403:0:(client.c:3082:ptlrpc_free_committed()) lustre-OST0000-osc-ffff9f14902b3800: committing for last_committed 21474840670 gen 1
00000008:00010000:1.0:1775764864.183243:0:2403:0:(osc_cache.c:901:osc_extent_finish()) ### no dlm lock: extent: ffff9f14a04c9240
00000020:00200000:1.0:1775764864.183251:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 255 items remaining.
00000020:00200000:1.0:1775764864.183254:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 254 items remaining.
00000020:00200000:1.0:1775764864.183256:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 253 items remaining.
00000020:00200000:1.0:1775764864.183258:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 252 items remaining.
00000020:00200000:1.0:1775764864.183260:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 251 items remaining.
00000020:00200000:1.0:1775764864.183262:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 250 items remaining.
00000020:00200000:1.0:1775764864.183263:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 249 items remaining.
00000020:00200000:1.0:1775764864.183266:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 248 items remaining.
00000020:00200000:1.0:1775764864.183268:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 247 items remaining.
00000020:00200000:1.0:1775764864.183269:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 246 items remaining.
00000020:00200000:1.0:1775764864.183271:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 245 items remaining.
00000020:00200000:1.0:1775764864.183273:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 244 items remaining.
00000020:00200000:1.0:1775764864.183275:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 243 items remaining.
00000020:00200000:1.0:1775764864.183276:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 242 items remaining.
00000020:00200000:1.0:1775764864.183278:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 241 items remaining.
00000020:00200000:1.0:1775764864.183279:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 240 items remaining.
00000020:00200000:1.0:1775764864.183281:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 239 items remaining.
00000020:00200000:1.0:1775764864.183283:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 238 items remaining.
00000020:00200000:1.0:1775764864.183284:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 237 items remaining.
00000020:00200000:1.0:1775764864.183286:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 236 items remaining.
00000020:00200000:1.0:1775764864.183288:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 235 items remaining.
00000020:00200000:1.0:1775764864.183289:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 234 items remaining.
00000020:00200000:1.0:1775764864.183291:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 233 items remaining.
00000020:00200000:1.0:1775764864.183292:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 232 items remaining.
00000020:00200000:1.0:1775764864.183294:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 231 items remaining.
00000020:00200000:1.0:1775764864.183296:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 230 items remaining.
00000020:00200000:1.0:1775764864.183297:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 229 items remaining.
00000020:00200000:1.0:1775764864.183299:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 228 items remaining.
00000020:00200000:1.0:1775764864.183300:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 227 items remaining.
00000020:00200000:1.0:1775764864.183302:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 226 items remaining.
00000020:00200000:1.0:1775764864.183304:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 225 items remaining.
00000020:00200000:1.0:1775764864.183306:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 224 items remaining.
00000020:00200000:1.0:1775764864.183307:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 223 items remaining.
00000020:00200000:1.0:1775764864.183309:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 222 items remaining.
00000020:00200000:1.0:1775764864.183310:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 221 items remaining.
00000020:00200000:1.0:1775764864.183312:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 220 items remaining.
00000020:00200000:1.0:1775764864.183313:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 219 items remaining.
00000020:00200000:1.0:1775764864.183315:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 218 items remaining.
00000020:00200000:1.0:1775764864.183317:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 217 items remaining.
00000020:00200000:1.0:1775764864.183319:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 216 items remaining.
00000020:00200000:1.0:1775764864.183321:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 215 items remaining.
00000020:00200000:1.0:1775764864.183323:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 214 items remaining.
00000020:00200000:1.0:1775764864.183324:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 213 items remaining.
00000020:00200000:1.0:1775764864.183326:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 212 items remaining.
00000020:00200000:1.0:1775764864.183327:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 211 items remaining.
00000020:00200000:1.0:1775764864.183328:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 210 items remaining.
00000020:00200000:1.0:1775764864.183330:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 209 items remaining.
00000020:00200000:1.0:1775764864.183331:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 208 items remaining.
00000020:00200000:1.0:1775764864.183333:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 207 items remaining.
00000020:00200000:1.0:1775764864.183334:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 206 items remaining.
00000020:00200000:1.0:1775764864.183336:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 205 items remaining.
00000020:00200000:1.0:1775764864.183338:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 204 items remaining.
00000020:00200000:1.0:1775764864.183340:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 203 items remaining.
00000020:00200000:1.0:1775764864.183341:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 202 items remaining.
00000020:00200000:1.0:1775764864.183344:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 201 items remaining.
00000020:00200000:1.0:1775764864.183346:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 200 items remaining.
00000020:00200000:1.0:1775764864.183347:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 199 items remaining.
00000020:00200000:1.0:1775764864.183349:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 198 items remaining.
00000020:00200000:1.0:1775764864.183350:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 197 items remaining.
00000020:00200000:1.0:1775764864.183352:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 196 items remaining.
00000020:00200000:1.0:1775764864.183353:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 195 items remaining.
00000020:00200000:1.0:1775764864.183355:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 194 items remaining.
00000020:00200000:1.0:1775764864.183357:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 193 items remaining.
00000020:00200000:1.0:1775764864.183359:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 192 items remaining.
00000020:00200000:1.0:1775764864.183361:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 191 items remaining.
00000020:00200000:1.0:1775764864.183362:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 190 items remaining.
00000020:00200000:1.0:1775764864.183364:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 189 items remaining.
00000020:00200000:1.0:1775764864.183366:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 188 items remaining.
00000020:00200000:1.0:1775764864.183367:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 187 items remaining.
00000020:00200000:1.0:1775764864.183369:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 186 items remaining.
00000020:00200000:1.0:1775764864.183371:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 185 items remaining.
00000020:00200000:1.0:1775764864.183373:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 184 items remaining.
00000020:00200000:1.0:1775764864.183375:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 183 items remaining.
00000020:00200000:1.0:1775764864.183376:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 182 items remaining.
00000020:00200000:1.0:1775764864.183378:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 181 items remaining.
00000020:00200000:1.0:1775764864.183380:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 180 items remaining.
00000020:00200000:1.0:1775764864.183381:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 179 items remaining.
00000020:00200000:1.0:1775764864.183383:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 178 items remaining.
00000020:00200000:1.0:1775764864.183384:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 177 items remaining.
00000020:00200000:1.0:1775764864.183386:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 176 items remaining.
00000020:00200000:1.0:1775764864.183387:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 175 items remaining.
00000020:00200000:1.0:1775764864.183389:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 174 items remaining.
00000020:00200000:1.0:1775764864.183391:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 173 items remaining.
00000020:00200000:1.0:1775764864.183393:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 172 items remaining.
00000020:00200000:1.0:1775764864.183395:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 171 items remaining.
00000020:00200000:1.0:1775764864.183397:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 170 items remaining.
00000020:00200000:1.0:1775764864.183399:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 169 items remaining.
00000020:00200000:1.0:1775764864.183401:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 168 items remaining.
00000020:00200000:1.0:1775764864.183402:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 167 items remaining.
00000020:00200000:1.0:1775764864.183405:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 166 items remaining.
00000020:00200000:1.0:1775764864.183407:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 165 items remaining.
00000020:00200000:1.0:1775764864.183409:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 164 items remaining.
00000020:00200000:1.0:1775764864.183410:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 163 items remaining.
00000020:00200000:1.0:1775764864.183412:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 162 items remaining.
00000020:00200000:1.0:1775764864.183414:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 161 items remaining.
00000020:00200000:1.0:1775764864.183416:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 160 items remaining.
00000020:00200000:1.0:1775764864.183417:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 159 items remaining.
00000020:00200000:1.0:1775764864.183418:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 158 items remaining.
00000020:00200000:1.0:1775764864.183420:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 157 items remaining.
00000020:00200000:1.0:1775764864.183421:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 156 items remaining.
00000020:00200000:1.0:1775764864.183424:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 155 items remaining.
00000020:00200000:1.0:1775764864.183425:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 154 items remaining.
00000020:00200000:1.0:1775764864.183427:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 153 items remaining.
00000020:00200000:1.0:1775764864.183429:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 152 items remaining.
00000020:00200000:1.0:1775764864.183431:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 151 items remaining.
00000020:00200000:1.0:1775764864.183433:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 150 items remaining.
00000020:00200000:1.0:1775764864.183446:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 149 items remaining.
00000020:00200000:1.0:1775764864.183448:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 148 items remaining.
00000020:00200000:1.0:1775764864.183449:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 147 items remaining.
00000020:00200000:1.0:1775764864.183451:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 146 items remaining.
00000020:00200000:1.0:1775764864.183452:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 145 items remaining.
00000020:00200000:1.0:1775764864.183454:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 144 items remaining.
00000020:00200000:1.0:1775764864.183455:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 143 items remaining.
00000020:00200000:1.0:1775764864.183456:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 142 items remaining.
00000020:00200000:1.0:1775764864.183458:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 141 items remaining.
00000020:00200000:1.0:1775764864.183460:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 140 items remaining.
00000020:00200000:1.0:1775764864.183462:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 139 items remaining.
00000020:00200000:1.0:1775764864.183464:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 138 items remaining.
00000020:00200000:1.0:1775764864.183466:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 137 items remaining.
00000020:00200000:1.0:1775764864.183468:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 136 items remaining.
00000020:00200000:1.0:1775764864.183469:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 135 items remaining.
00000020:00200000:1.0:1775764864.183471:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 134 items remaining.
00000020:00200000:1.0:1775764864.183473:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 133 items remaining.
00000020:00200000:1.0:1775764864.183475:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 132 items remaining.
00000020:00200000:1.0:1775764864.183476:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 131 items remaining.
00000020:00200000:1.0:1775764864.183485:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 130 items remaining.
00000020:00200000:1.0:1775764864.183487:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 129 items remaining.
00000020:00200000:1.0:1775764864.183489:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 128 items remaining.
00000020:00200000:1.0:1775764864.183491:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 127 items remaining.
00000020:00200000:1.0:1775764864.183492:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 126 items remaining.
00000020:00200000:1.0:1775764864.183494:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 125 items remaining.
00000020:00200000:1.0:1775764864.183496:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 124 items remaining.
00000020:00200000:1.0:1775764864.183497:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 123 items remaining.
00000020:00200000:1.0:1775764864.183499:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 122 items remaining.
00000020:00200000:1.0:1775764864.183501:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 121 items remaining.
00000020:00200000:1.0:1775764864.183503:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 120 items remaining.
00000020:00200000:1.0:1775764864.183504:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 119 items remaining.
00000020:00200000:1.0:1775764864.183506:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 118 items remaining.
00000020:00200000:1.0:1775764864.183508:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 117 items remaining.
00000020:00200000:1.0:1775764864.183510:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 116 items remaining.
00000020:00200000:1.0:1775764864.183511:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 115 items remaining.
00000020:00200000:1.0:1775764864.183513:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 114 items remaining.
00000020:00200000:1.0:1775764864.183514:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 113 items remaining.
00000020:00200000:1.0:1775764864.183516:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 112 items remaining.
00000020:00200000:1.0:1775764864.183518:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 111 items remaining.
00000020:00200000:1.0:1775764864.183519:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 110 items remaining.
00000020:00200000:1.0:1775764864.183521:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 109 items remaining.
00000020:00200000:1.0:1775764864.183523:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 108 items remaining.
00000020:00200000:1.0:1775764864.183525:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 107 items remaining.
00000020:00200000:1.0:1775764864.183526:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 106 items remaining.
00000020:00200000:1.0:1775764864.183528:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 105 items remaining.
00000020:00200000:1.0:1775764864.183530:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 104 items remaining.
00000020:00200000:1.0:1775764864.183531:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 103 items remaining.
00000020:00200000:1.0:1775764864.183533:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 102 items remaining.
00000020:00200000:1.0:1775764864.183534:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 101 items remaining.
00000020:00200000:1.0:1775764864.183536:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 100 items remaining.
00000020:00200000:1.0:1775764864.183537:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 99 items remaining.
00000020:00200000:1.0:1775764864.183538:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 98 items remaining.
00000020:00200000:1.0:1775764864.183540:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 97 items remaining.
00000020:00200000:1.0:1775764864.183541:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 96 items remaining.
00000020:00200000:1.0:1775764864.183543:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 95 items remaining.
00000020:00200000:1.0:1775764864.183544:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 94 items remaining.
00000020:00200000:1.0:1775764864.183546:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 93 items remaining.
00000020:00200000:1.0:1775764864.183548:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 92 items remaining.
00000020:00200000:1.0:1775764864.183549:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 91 items remaining.
00000020:00200000:1.0:1775764864.183551:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 90 items remaining.
00000020:00200000:1.0:1775764864.183588:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 89 items remaining.
00000020:00200000:1.0:1775764864.183591:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 88 items remaining.
00000020:00200000:1.0:1775764864.183592:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 87 items remaining.
00000020:00200000:1.0:1775764864.183594:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 86 items remaining.
00000020:00200000:1.0:1775764864.183595:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 85 items remaining.
00000020:00200000:1.0:1775764864.183597:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 84 items remaining.
00000020:00200000:1.0:1775764864.183599:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 83 items remaining.
00000020:00200000:1.0:1775764864.183600:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 82 items remaining.
00000020:00200000:1.0:1775764864.183602:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 81 items remaining.
00000020:00200000:1.0:1775764864.183603:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 80 items remaining.
00000020:00200000:1.0:1775764864.183605:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 79 items remaining.
00000020:00200000:1.0:1775764864.183606:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 78 items remaining.
00000020:00200000:1.0:1775764864.183608:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 77 items remaining.
00000020:00200000:1.0:1775764864.183610:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 76 items remaining.
00000020:00200000:1.0:1775764864.183612:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 75 items remaining.
00000020:00200000:1.0:1775764864.183613:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 74 items remaining.
00000020:00200000:1.0:1775764864.183615:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 73 items remaining.
00000020:00200000:1.0:1775764864.183616:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 72 items remaining.
00000020:00200000:1.0:1775764864.183617:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 71 items remaining.
00000020:00200000:1.0:1775764864.183619:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 70 items remaining.
00000020:00200000:1.0:1775764864.183620:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 69 items remaining.
00000020:00200000:1.0:1775764864.183622:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 68 items remaining.
00000020:00200000:1.0:1775764864.183623:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 67 items remaining.
00000020:00200000:1.0:1775764864.183625:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 66 items remaining.
00000020:00200000:1.0:1775764864.183626:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 65 items remaining.
00000020:00200000:1.0:1775764864.183628:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 64 items remaining.
00000020:00200000:1.0:1775764864.183629:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 63 items remaining.
00000020:00200000:1.0:1775764864.183631:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 62 items remaining.
00000020:00200000:1.0:1775764864.183633:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 61 items remaining.
00000020:00200000:1.0:1775764864.183635:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 60 items remaining.
00000020:00200000:1.0:1775764864.183636:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 59 items remaining.
00000020:00200000:1.0:1775764864.183638:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 58 items remaining.
00000020:00200000:1.0:1775764864.183640:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 57 items remaining.
00000020:00200000:1.0:1775764864.183642:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 56 items remaining.
00000020:00200000:1.0:1775764864.183643:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 55 items remaining.
00000020:00200000:1.0:1775764864.183645:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 54 items remaining.
00000020:00200000:1.0:1775764864.183647:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 53 items remaining.
00000020:00200000:1.0:1775764864.183649:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 52 items remaining.
00000020:00200000:1.0:1775764864.183650:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 51 items remaining.
00000020:00200000:1.0:1775764864.183651:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 50 items remaining.
00000020:00200000:1.0:1775764864.183653:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 49 items remaining.
00000020:00200000:1.0:1775764864.183654:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 48 items remaining.
00000020:00200000:1.0:1775764864.183656:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 47 items remaining.
00000020:00200000:1.0:1775764864.183658:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 46 items remaining.
00000020:00200000:1.0:1775764864.183659:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 45 items remaining.
00000020:00200000:1.0:1775764864.183661:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 44 items remaining.
00000020:00200000:1.0:1775764864.183663:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 43 items remaining.
00000020:00200000:1.0:1775764864.183675:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 42 items remaining.
00000020:00200000:1.0:1775764864.183678:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 41 items remaining.
00000020:00200000:1.0:1775764864.183679:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 40 items remaining.
00000020:00200000:1.0:1775764864.183681:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 39 items remaining.
00000020:00200000:1.0:1775764864.183682:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 38 items remaining.
00000020:00200000:1.0:1775764864.183684:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 37 items remaining.
00000020:00200000:1.0:1775764864.183685:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 36 items remaining.
00000020:00200000:1.0:1775764864.183687:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 35 items remaining.
00000020:00200000:1.0:1775764864.183689:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 34 items remaining.
00000020:00200000:1.0:1775764864.183690:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 33 items remaining.
00000020:00200000:1.0:1775764864.183692:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 32 items remaining.
00000020:00200000:1.0:1775764864.183693:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 31 items remaining.
00000020:00200000:1.0:1775764864.183695:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 30 items remaining.
00000020:00200000:1.0:1775764864.183697:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 29 items remaining.
00000020:00200000:1.0:1775764864.183698:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 28 items remaining.
00000020:00200000:1.0:1775764864.183700:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 27 items remaining.
00000020:00200000:1.0:1775764864.183701:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 26 items remaining.
00000020:00200000:1.0:1775764864.183703:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 25 items remaining.
00000020:00200000:1.0:1775764864.183705:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 24 items remaining.
00000020:00200000:1.0:1775764864.183706:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 23 items remaining.
00000020:00200000:1.0:1775764864.183708:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 22 items remaining.
00000020:00200000:1.0:1775764864.183710:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 21 items remaining.
00000020:00200000:1.0:1775764864.183712:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 20 items remaining.
00000020:00200000:1.0:1775764864.183713:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 19 items remaining.
00000020:00200000:1.0:1775764864.183715:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 18 items remaining.
00000020:00200000:1.0:1775764864.183716:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 17 items remaining.
00000020:00200000:1.0:1775764864.183718:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 16 items remaining.
00000020:00200000:1.0:1775764864.183720:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 15 items remaining.
00000020:00200000:1.0:1775764864.183722:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 14 items remaining.
00000020:00200000:1.0:1775764864.183723:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 13 items remaining.
00000020:00200000:1.0:1775764864.183725:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 12 items remaining.
00000020:00200000:1.0:1775764864.183727:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 11 items remaining.
00000020:00200000:1.0:1775764864.183729:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 10 items remaining.
00000020:00200000:1.0:1775764864.183730:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 9 items remaining.
00000020:00200000:1.0:1775764864.183732:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 8 items remaining.
00000020:00200000:1.0:1775764864.183734:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 7 items remaining.
00000020:00200000:1.0:1775764864.183736:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 6 items remaining.
00000020:00200000:1.0:1775764864.183737:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 5 items remaining.
00000020:00200000:1.0:1775764864.183739:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 4 items remaining.
00000020:00200000:1.0:1775764864.183740:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 3 items remaining.
00000020:00200000:1.0:1775764864.183742:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 2 items remaining.
00000020:00200000:1.0:1775764864.183744:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining.
00000020:00200000:1.0:1775764864.183746:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining.
00000020:00200000:1.0:1775764864.183834:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining.
00000008:00010000:1.0:1775764864.183856:0:2403:0:(osc_cache.c:496:__osc_extent_remove()) ### no dlm lock: extent: ffff9f14a04c9240
00000100:00100000:1.0:1775764864.184694:0:2403:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2403:1862022114166912:192.168.202.110@tcp:4:lfs.0
00000100:00100000:1.0:1775764864.268184:0:2395:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.110@tcp (source: 12345-192.168.202.110@tcp)
00000100:00100000:1.0:1775764864.276010:0:2395:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.110@tcp (source: 12345-192.168.202.110@tcp)
00000100:00100000:1.0:1775764864.305825:0:2396:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.110@tcp (source: 12345-192.168.202.110@tcp)
00010000:00010000:1.0:1775764864.323095:0:72291:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f14ab59be00/0x23cab7dad48e6197 lrc: 2/1,0 mode: CR/CR res: [0x200000405:0x7b:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x0 nid: local remote: 0x9293047872332ac expref: -99 pid: 72291 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0
00010000:00010000:1.0:1775764864.323112:0:72291:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f14ab59be00/0x23cab7dad48e6197 lrc: 2/1,0 mode: CR/CR res: [0x200000405:0x7b:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x9293047872332ac expref: -99 pid: 72291 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0
00000080:00010000:1.0:1775764864.323123:0:72291:0:(file.c:6871:ll_layout_lock_set()) ### file [0x200000405:0x7b:0x0](0000000021680a44) being reconfigured ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f14ab59be00/0x23cab7dad48e6197 lrc: 3/1,0 mode: CR/CR res: [0x200000405:0x7b:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x9293047872332ac expref: -99 pid: 72291 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0
00000080:00200000:1.0:1775764864.323218:0:72291:0:(file.c:6766:ll_layout_conf()) [0x200000405:0x7b:0x0]: layout version change: 4294967294 -> 3
00010000:00010000:1.0:1775764864.323244:0:72291:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f14ab59be00/0x23cab7dad48e6197 lrc: 3/1,0 mode: CR/CR res: [0x200000405:0x7b:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x9293047872332ac expref: -99 pid: 72291 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0
00010000:00010000:1.0:1775764864.323253:0:72291:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f14ab59be00/0x23cab7dad48e6197 lrc: 2/0,0 mode: CR/CR res: [0x200000405:0x7b:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x9293047872332ac expref: -99 pid: 72291 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0
00020000:00200000:1.0:1775764864.323275:0:72291:0:(lov_io.c:458:lov_io_mirror_init()) [0x200000405:0x7b:0x0]: flr state: 2, move mirror from 0 to 0, have retried: 0, mirror count: 2
00020000:00200000:1.0:1775764864.323278:0:72291:0:(lov_io.c:484:lov_io_mirror_init()) use delayed RPC state for this IO
00020000:00200000:1.0:1775764864.323287:0:72291:0:(lov_io.c:860:lov_io_iter_init()) component[0] flags 0x10
00000080:00200000:1.0:1775764864.323328:0:72291:0:(vvp_io.c:198:vvp_io_one_lock_index()) lock: 1 [0, 18446744073709551615]
00000008:00010000:1.0:1775764864.323360:0:72291:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 0
00000008:00010000:1.0:1775764864.323364:0:72291:0:(osc_lock.c:1310:osc_lock_init()) ### lock ffff9f14899bbe70, osc lock ffff9f148759bb50, flags 0x0
00000008:00010000:1.0:1775764864.323368:0:72291:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 0
00000008:00010000:1.0:1775764864.323369:0:72291:0:(osc_lock.c:1310:osc_lock_init()) ### lock ffff9f14899bbeb0, osc lock ffff9f148759a130, flags 0x0
00010000:00010000:1.0:1775764864.323898:0:72291:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9f14a0487c00 type 11 mode 2 res 11811161089/185 (0 18446744073709551615)
00010000:00010000:1.0:1775764864.323942:0:72291:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-OST0001-osc-ffff9f14902b3800 lock: 00000000d6631618/0x23cab7dad48e619e lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0xb9:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 72291 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0
00010000:00010000:1.0:1775764864.323953:0:72291:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x0 ns: lustre-OST0001-osc-ffff9f14902b3800 lock: 00000000d6631618/0x23cab7dad48e619e lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0xb9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 72291 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0
00010000:00010000:1.0:1775764864.323969:0:72291:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-OST0001-osc-ffff9f14902b3800 lock: 00000000d6631618/0x23cab7dad48e619e lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0xb9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 72291 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0
00000100:00100000:1.0:1775764864.323996:0:72291:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job lfs:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:72291:1862022114169600:192.168.202.110@tcp:101:lfs.0
00000100:00100000:1.0:1775764864.324118:0:72291:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14899bb0c0 going to sleep for 11 seconds
00000100:00100000:1.0:1775764864.326881:0:2396:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.110@tcp (source: 12345-192.168.202.110@tcp)
00000100:00100000:1.0:1775764864.337148:0:2396:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.110@tcp (source: 12345-192.168.202.110@tcp)
00000100:00100000:1.0:1775764864.347985:0:2396:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.110@tcp (source: 12345-192.168.202.110@tcp)
00000020:00010000:1.0:1775764864.359285:0:72291:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff9f14861245d8 (ffff9f1486143638/0) at cl_lock_release():245
00000020:00010000:1.0:1775764864.359289:0:72291:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9f14861245d8 (ffff9f1486143638/0) at cl_lock_cancel():142
00000020:00010000:1.0:1775764864.359295:0:72291:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9f14899bbe70 (ffff9f1487c502d8/1) at cl_lock_cancel():142
00010000:00010000:1.0:1775764864.359305:0:72291:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-OST0001-osc-ffff9f14902b3800 lock: 00000000d6631618/0x23cab7dad48e619e lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0xb9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x810020000020000 nid: local remote: 0x9293047872332b3 expref: -99 pid: 72291 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0
00010000:00010000:1.0:1775764864.359323:0:72291:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-OST0001-osc-ffff9f14902b3800 lock: 00000000d6631618/0x23cab7dad48e619e lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0xb9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800020000020000 nid: local remote: 0x9293047872332b3 expref: -99 pid: 72291 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0
00000020:00010000:1.0:1775764864.359340:0:72291:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9f14899bbeb0 (ffff9f1487c50128/1) at cl_lock_cancel():142
00010000:00010000:1.0:1775764864.359345:0:72291:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-OST0000-osc-ffff9f14902b3800 lock: 00000000f598f796/0x23cab7dad48e61a5 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0xb8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x810020000020000 nid: local remote: 0x9293047872332ba expref: -99 pid: 72291 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0
00010000:00010000:1.0:1775764864.359352:0:72291:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-OST0000-osc-ffff9f14902b3800 lock: 00000000f598f796/0x23cab7dad48e61a5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0xb8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800020000020000 nid: local remote: 0x9293047872332ba expref: -99 pid: 72291 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0
00000020:00010000:1.0:1775764864.359367:0:72291:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9f14861245d8 (ffff9f1486143638/0) at cl_lock_fini():72
00000020:00010000:1.0:1775764864.359369:0:72291:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9f14899bbe70 (ffff9f1486143638/1) at cl_lock_fini():72
00000020:00010000:1.0:1775764864.359374:0:72291:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9f14899bbeb0 (ffff9f1486143638/1) at cl_lock_fini():72
00000080:00200000:1.0:1775764864.359422:0:72291:0:(vvp_io.c:288:vvp_io_fini()) [0x200000405:0x7b:0x0] ignore/verify layout 0/0, layout version 3 need write layout 0, restore needed 0, invalidate_lock 0
00000080:00200008:1.0:1775764864.359447:0:72291:0:(llite_lib.c:2524:ll_setattr_raw()) COMPLETED file f207.sanity-flr:[0x200000405:0x7b:0x0](ffff9f14a1111148) current size 1048576, valid attrs 101f0, mode 9f14, uid 0, gid 0, new size 0, atime 1775764863.000000000, mtime 1775764862.000000000, ctime 1775764864.000000000, rc 0
00000080:00200080:1.0:1775764864.359474:0:72291:0:(file.c:4602:ll_file_ioctl()) VFS Op:inode=[0x200000405:0x7b:0x0](ffff9f14a1111148) cmd=c00c66f3 arg=d6a470
00010000:00010000:1.0:1775764864.359980:0:72291:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f149817fc00/0x23cab7dad48e6166 lrc: 2/0,0 mode: PW/PW res: [0x200000405:0x7b:0x0].0x0 bits 0x4/0x0 rrc: 4 type: IBT gid 0 flags: 0x81008000000000 nid: local remote: 0x92930478723322e expref: -99 pid: 72291 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0
00010000:00010000:1.0:1775764864.359997:0:72291:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f149817fc00/0x23cab7dad48e6166 lrc: 0/0,0 mode: --/PW res: [0x200000405:0x7b:0x0].0x0 bits 0x4/0x0 rrc: 4 type: IBT gid 0 flags: 0x85809400000000 nid: local remote: 0x92930478723322e expref: -99 pid: 72291 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0
00000002:00100000:1.0:1775764864.360017:0:72291:0:(mdc_request.c:964:mdc_close()) @@@ matched open  req@ffff9f14ab4b5880 x1862022114165504/t8589936674(8589936674) o101->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 808/896 e 0 to 0 dl 1775764918 ref 1 fl Complete:RPQU/604/ffffffff rc 0/-1 job:'lfs.0' uid:0 gid:0 projid:0
00000020:00100000:1.0:1775764864.360028:0:72291:0:(genops.c:2442:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9f14902b3800: modify RPC slot 1 is allocated opc 35, max 7
00000100:00100000:1.0:1775764864.360033:0:72291:0:(client.c:806:ptlrpc_reassign_next_xid()) @@@ reassign xid  req@ffff9f14ab4ff800 x1862022114170496/t0(0) o35->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:23/10 lens 504/2128 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0
00000100:00100000:1.0:1775764864.360059:0:72291:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff800 pname:cluuid:pid:xid:nid:opc:job lfs:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:72291:1862022114170496:192.168.202.110@tcp:35:lfs.0
00000100:00100000:1.0:1775764864.360260:0:72291:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14899bbe40 going to sleep for 11 seconds
00000100:00100000:1.0:1775764864.364226:0:39304:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1862022124542848
00000100:00100000:1.0:1775764864.364235:0:39304:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1862022124542848
00000100:00100000:1.0:1775764864.364287:0:39304:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1862022124542976
00000100:00100000:1.0:1775764864.364290:0:39304:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1862022124542976
00000100:00100000:1.0:1775764864.364306:0:39304:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.110@tcp, seq: 6429
00000100:00100000:1.0:1775764864.364316:0:39304:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9f14a04d0000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48+6:35251:x1862022124542848:12345-192.168.202.110@tcp:104:
00010000:00010000:1.0:1775764864.364335:0:39304:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast  ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f14ab59be00/0x23cab7dad48e6197 lrc: 2/0,0 mode: CR/CR res: [0x200000405:0x7b:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x420000000000 nid: local remote: 0x9293047872332ac expref: -99 pid: 72291 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0
00010000:00010000:1.0:1775764864.364422:0:39304:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool
00000100:00100000:1.0:1775764864.364429:0:39304:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9f14a04d0000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48+6:35251:x1862022124542848:12345-192.168.202.110@tcp:104: Request processed in 114us (965us total) trans 0 rc 0/0
00000100:00100000:1.0:1775764864.364436:0:39304:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.110@tcp, seq: 6429
00000100:00100000:1.0:1775764864.364453:0:39304:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.110@tcp, seq: 6430
00000100:00100000:1.0:1775764864.364459:0:39304:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9f14a04d0380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48+5:35251:x1862022124542976:12345-192.168.202.110@tcp:104:
00010000:00010000:1.0:1775764864.364466:0:39304:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast  ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f1498178e00/0x23cab7dad48e60fd lrc: 2/0,0 mode: PR/PR res: [0x200000405:0x7b:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x929304787233108 expref: -99 pid: 72275 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0
00010000:00010000:1.0:1775764864.364512:0:39304:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to priority blp list, 2 blwis in pool
00000100:00100000:1.0:1775764864.364517:0:39304:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9f14a04d0380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48+5:35251:x1862022124542976:12345-192.168.202.110@tcp:104: Request processed in 59us (520us total) trans 0 rc 0/0
00000100:00100000:1.0:1775764864.364523:0:39304:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.110@tcp, seq: 6430
00000100:00100000:1.0:1775764864.374227:0:2395:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.110@tcp (source: 12345-192.168.202.110@tcp)
00000100:00100000:1.0:1775764864.374265:0:39304:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1862022124543104
00000100:00100000:1.0:1775764864.374273:0:39304:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1862022124543104
00000100:00100000:1.0:1775764864.374316:0:39304:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.110@tcp, seq: 6431
00000100:00100000:1.0:1775764864.374326:0:39304:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9f14ab4fd880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48+5:35251:x1862022124543104:12345-192.168.202.110@tcp:104:
00010000:00010000:1.0:1775764864.374349:0:39304:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast  ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f149817e000/0x23cab7dad48e6182 lrc: 2/0,0 mode: CR/CR res: [0x200000405:0x7d:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x420000000000 nid: local remote: 0x929304787233289 expref: -99 pid: 72291 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0
00010000:00010000:1.0:1775764864.374430:0:39304:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool
00000100:00100000:1.0:1775764864.374437:0:39304:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9f14ab4fd880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48+5:35251:x1862022124543104:12345-192.168.202.110@tcp:104: Request processed in 113us (217us total) trans 0 rc 0/0
00000100:00100000:1.0:1775764864.374446:0:39304:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.110@tcp, seq: 6431
00000100:00100000:1.0:1775764864.927147:0:39304:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1862022124544256
00000100:00100000:1.0:1775764864.927158:0:39304:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1862022124544256
00000100:00100000:1.0:1775764864.927252:0:39304:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.110@tcp, seq: 6432
00000100:00100000:1.0:1775764864.927262:0:39304:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9f14a04d0e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:LOV_OSC_UUID+4:8372:x1862022124544256:12345-192.168.202.110@tcp:104:
00010000:00010000:1.0:1775764864.927280:0:39304:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast  ns: lustre-OST0001-osc-ffff9f14902b3800 lock: 00000000d6631618/0x23cab7dad48e619e lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0xb9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800420000020000 nid: local remote: 0x9293047872332b3 expref: -99 pid: 72291 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0
00010000:00010000:1.0:1775764864.927359:0:39304:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool
00000100:00100000:1.0:1775764864.927366:0:39304:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9f14a04d0e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:LOV_OSC_UUID+4:8372:x1862022124544256:12345-192.168.202.110@tcp:104: Request processed in 105us (539us total) trans 0 rc 0/0
00000100:00100000:1.0:1775764864.927373:0:39304:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.110@tcp, seq: 6432
00000100:00100000:1.0:1775764864.954113:0:2396:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.110@tcp (source: 12345-192.168.202.110@tcp)
00000100:00100000:1.0:1775764864.954161:0:2396:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.110@tcp (source: 12345-192.168.202.110@tcp)
00000100:00100000:1.0:1775764864.954202:0:44145:0:(service.c:153:ptlrpc_grow_req_bufs()) ldlm_cbd: allocate 1 new 8192-byte reqbufs (0/2 left), rc = 0
00000100:00100000:1.0:1775764864.954220:0:44145:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1862022124544384
00000100:00100000:1.0:1775764864.954229:0:44145:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1862022124544384
00000100:00100000:1.0:1775764864.954268:0:44145:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1862022124544512
00000100:00100000:1.0:1775764864.954270:0:44145:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1862022124544512
00000100:00100000:1.0:1775764864.954281:0:44145:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.110@tcp, seq: 6433
00000100:00100000:1.0:1775764864.954288:0:44145:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9f14ab4ff480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:LOV_OSC_UUID+4:36469:x1862022124544384:12345-192.168.202.110@tcp:106:
00010000:00010000:1.0:1775764864.954300:0:44145:0:(ldlm_lockd.c:2525:ldlm_callback_handler()) ### glimpse ast  ns: lustre-OST0000-osc-ffff9f14902b3800 lock: 0000000039155643/0x23cab7dad48e6189 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0xba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800020000000000 nid: local remote: 0x929304787233290 expref: -99 pid: 72291 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0
00010000:00010000:1.0:1775764864.954310:0:44145:0:(ldlm_lockd.c:2116:ldlm_handle_gl_callback()) ### client glimpse AST callback handler ns: lustre-OST0000-osc-ffff9f14902b3800 lock: 0000000039155643/0x23cab7dad48e6189 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0xba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800020000000000 nid: local remote: 0x929304787233290 expref: -99 pid: 72291 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0
00000020:00010000:1.0:1775764864.954353:0:44145:0:(cl_object.c:330:cl_object_glimpse()) header@00000000e99680f2[0x110, 1, [0x200000405:0x7d:0x0] hash]
00000020:00010000:1.0:1775764864.954356:0:44145:0:(cl_object.c:330:cl_object_glimpse()) size: 1048576 mtime: 1775764862 atime: 1775764863 ctime: 1775764864 blocks: 2048
00000020:00010000:1.0:1775764864.954359:0:44145:0:(cl_object.c:330:cl_object_glimpse()) header@0000000053ab490e[0x116, 2, [0x280000401:0xba:0x0]]
00000020:00010000:1.0:1775764864.954361:0:44145:0:(cl_object.c:330:cl_object_glimpse()) size: 1048576 mtime: 1775764862 atime: 1775764863 ctime: 1775764864 blocks: 2048
00000100:00100000:1.0:1775764864.954425:0:44145:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9f14ab4ff480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:LOV_OSC_UUID+4:36469:x1862022124544384:12345-192.168.202.110@tcp:106: Request processed in 137us (323us total) trans 0 rc 0/0
00000100:00100000:1.0:1775764864.954431:0:44145:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.110@tcp, seq: 6433
00000100:00100000:1.0:1775764864.954445:0:44145:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.110@tcp, seq: 6434
00000100:00100000:1.0:1775764864.954451:0:44145:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9f1484cfb450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:LOV_OSC_UUID+4:27790:x1862022124544512:12345-192.168.202.110@tcp:106:
00010000:00010000:1.0:1775764864.954458:0:44145:0:(ldlm_lockd.c:2525:ldlm_callback_handler()) ### glimpse ast  ns: lustre-OST0001-osc-ffff9f14902b3800 lock: 00000000c8e2fd2d/0x23cab7dad48e6190 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0xbb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800020000020000 nid: local remote: 0x929304787233297 expref: -99 pid: 72291 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0
00010000:00010000:1.0:1775764864.954465:0:44145:0:(ldlm_lockd.c:2116:ldlm_handle_gl_callback()) ### client glimpse AST callback handler ns: lustre-OST0001-osc-ffff9f14902b3800 lock: 00000000c8e2fd2d/0x23cab7dad48e6190 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0xbb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800020000020000 nid: local remote: 0x929304787233297 expref: -99 pid: 72291 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0
00000020:00010000:1.0:1775764864.954488:0:44145:0:(cl_object.c:330:cl_object_glimpse()) header@00000000e99680f2[0x110, 1, [0x200000405:0x7d:0x0] hash]
00000020:00010000:1.0:1775764864.954490:0:44145:0:(cl_object.c:330:cl_object_glimpse()) size: 0 mtime: 1775764862 atime: 1775764863 ctime: 1775764864 blocks: 0
00000020:00010000:1.0:1775764864.954494:0:44145:0:(cl_object.c:330:cl_object_glimpse()) header@000000009b334017[0x116, 2, [0x2c0000401:0xbb:0x0]]
00000020:00010000:1.0:1775764864.954495:0:44145:0:(cl_object.c:330:cl_object_glimpse()) size: 0 mtime: 1775764862 atime: 1775764863 ctime: 1775764864 blocks: 0
00000100:00100000:1.0:1775764864.954520:0:44145:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9f1484cfb450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:LOV_OSC_UUID+4:27790:x1862022124544512:12345-192.168.202.110@tcp:106: Request processed in 69us (357us total) trans 0 rc 0/0
00000100:00100000:1.0:1775764864.954526:0:44145:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.110@tcp, seq: 6434
00000100:00100000:1.0:1775764864.963067:0:2396:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.110@tcp (source: 12345-192.168.202.110@tcp)
00000100:00100000:1.0:1775764865.125300:0:2403:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2403:1862022114174848:192.168.202.110@tcp:3:cmp.0
00000008:00010000:1.0:1775764865.142876:0:2403:0:(osc_cache.c:901:osc_extent_finish()) ### no dlm lock: extent: ffff9f14852d8300
00000020:00200000:1.0:1775764865.142901:0:2403:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining.
00000008:00010000:1.0:1775764865.143005:0:2403:0:(osc_cache.c:496:__osc_extent_remove()) ### no dlm lock: extent: ffff9f14852d8300
00000100:00100000:1.0:1775764865.143031:0:2403:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2403:1862022114174848:192.168.202.110@tcp:3:cmp.0
00000100:00100000:1.0:1775764865.150428:0:2404:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2404:1862022114174976:192.168.202.110@tcp:3:cmp.0
00000008:00010000:1.0:1775764865.153782:0:2404:0:(osc_cache.c:901:osc_extent_finish()) ### no dlm lock: extent: ffff9f14852d8180
00000020:00200000:1.0:1775764865.153797:0:2404:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining.
00000008:00010000:1.0:1775764865.153857:0:2404:0:(osc_cache.c:496:__osc_extent_remove()) ### no dlm lock: extent: ffff9f14852d8180
00000100:00100000:1.0:1775764865.153897:0:2404:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2404:1862022114174976:192.168.202.110@tcp:3:cmp.0
00000100:00100000:1.0:1775764865.158602:0:39304:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1862022124544896
00000100:00100000:1.0:1775764865.158615:0:39304:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1862022124544896
00000100:00100000:1.0:1775764865.158680:0:39304:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.110@tcp, seq: 6437
00000100:00100000:1.0:1775764865.158691:0:39304:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9f14a04d1500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:LOV_OSC_UUID+4:27788:x1862022124544896:12345-192.168.202.110@tcp:106:
00010000:00010000:1.0:1775764865.158711:0:39304:0:(ldlm_lockd.c:2525:ldlm_callback_handler()) ### glimpse ast  ns: lustre-OST0001-osc-ffff9f14902b3800 lock: 00000000c8e2fd2d/0x23cab7dad48e6190 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0xbb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800020000020000 nid: local remote: 0x929304787233297 expref: -99 pid: 72291 timeout: 0 lvb_type: 1 lru_score: 4 lru_type: 0
00010000:00010000:1.0:1775764865.158726:0:39304:0:(ldlm_lockd.c:2116:ldlm_handle_gl_callback()) ### client glimpse AST callback handler ns: lustre-OST0001-osc-ffff9f14902b3800 lock: 00000000c8e2fd2d/0x23cab7dad48e6190 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0xbb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800020000020000 nid: local remote: 0x929304787233297 expref: -99 pid: 72291 timeout: 0 lvb_type: 1 lru_score: 4 lru_type: 0
00000020:00010000:1.0:1775764865.158781:0:39304:0:(cl_object.c:330:cl_object_glimpse()) header@00000000e99680f2[0x110, 1, [0x200000405:0x7d:0x0] hash]
00000020:00010000:1.0:1775764865.158784:0:39304:0:(cl_object.c:330:cl_object_glimpse()) size: 0 mtime: 1775764862 atime: 1775764863 ctime: 1775764864 blocks: 0
00000020:00010000:1.0:1775764865.158788:0:39304:0:(cl_object.c:330:cl_object_glimpse()) header@000000009b334017[0x116, 2, [0x2c0000401:0xbb:0x0]]
00000020:00010000:1.0:1775764865.158790:0:39304:0:(cl_object.c:330:cl_object_glimpse()) size: 0 mtime: 1775764862 atime: 1775764863 ctime: 1775764864 blocks: 0
00000100:00100000:1.0:1775764865.158880:0:39304:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9f14a04d1500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:LOV_OSC_UUID+4:27788:x1862022124544896:12345-192.168.202.110@tcp:106: Request processed in 180us (401us total) trans 0 rc 0/0
00000100:00100000:1.0:1775764865.158892:0:39304:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.110@tcp, seq: 6437
00010000:00010000:1.0:1775764868.363383:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00010000:00010000:1.0:1775764871.435678:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000002:00010000:1.0:1775764871.908612:0:72725:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0
00010000:00010000:1.0:1775764871.908644:0:72725:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f149817e000/0x23cab7dad48e61b3 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x9293047872332e4 expref: -99 pid: 72293 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 0
00010000:00010000:1.0:1775764871.908661:0:72725:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f149817e000/0x23cab7dad48e61b3 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x9293047872332e4 expref: -99 pid: 72293 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 0
00000080:00200000:1.0:1775764871.908669:0:72725:0:(namei.c:953:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1
00000080:00010000:1.0:1775764871.908676:0:72725:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9f1491deba88)
00000080:00010000:1.0:1775764871.908680:0:72725:0:(llite_internal.h:1984:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9f1491deba88) for lock 0x23cab7dad48e61b3
00000080:00010000:1.0:1775764871.909280:0:72725:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x23cab7dad48e61b3 from it ffffb06c01d47b58
00010000:00010000:1.0:1775764871.909288:0:72725:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f149817e000/0x23cab7dad48e61b3 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x9293047872332e4 expref: -99 pid: 72293 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 0
00010000:00010000:1.0:1775764871.909299:0:72725:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f149817e000/0x23cab7dad48e61b3 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x9293047872332e4 expref: -99 pid: 72293 timeout: 0 lvb_type: 0 lru_score: 7 lru_type: 0
00000080:00200000:1.0:1775764871.909320:0:72725:0:(file.c:6593:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9f1491deba88), inode mode 41ed mask 1
00000080:00200000:1.0:1775764871.909333:0:72725:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=f207.sanity-flr-tt, flags=8
00000080:00200008:1.0:1775764871.909348:0:72725:0:(file.c:6226:ll_getattr_dentry()) START file f207.sanity-flr-tt:[0x200000405:0x7a:0x0](ffff9f14a10b00c8), request_mask 2047, flags 0, foreign 0
00000080:00200000:1.0:1775764871.909354:0:72725:0:(file.c:6121:ll_inode_revalidate()) VFS Op:inode=[0x200000405:0x7a:0x0](ffff9f14a10b00c8),name=f207.sanity-flr-tt
00000002:00010000:1.0:1775764871.909375:0:72725:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: ,[0x200000405:0x7a:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0
00010000:00010000:1.0:1775764871.909385:0:72725:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f149153e400/0x23cab7dad48e60b7 lrc: 2/1,0 mode: PR/PR res: [0x200000405:0x7a:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x929304787233021 expref: -99 pid: 72267 timeout: 0 lvb_type: 3 lru_score: 8 lru_type: 0
00010000:00010000:1.0:1775764871.909401:0:72725:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f149153e400/0x23cab7dad48e60b7 lrc: 2/1,0 mode: PR/PR res: [0x200000405:0x7a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x929304787233021 expref: -99 pid: 72267 timeout: 0 lvb_type: 3 lru_score: 8 lru_type: 0
00000080:00200000:1.0:1775764871.909407:0:72725:0:(namei.c:953:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1
00000080:00010000:1.0:1775764871.909412:0:72725:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000405:0x7a:0x0](ffff9f14a10b00c8)
00000080:00010000:1.0:1775764871.909415:0:72725:0:(llite_internal.h:1984:ll_set_lock_data()) setting l_data to inode [0x200000405:0x7a:0x0](ffff9f14a10b00c8) for lock 0x23cab7dad48e60b7
00000080:00010000:1.0:1775764871.909419:0:72725:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x23cab7dad48e60b7 from it ffffb06c01d47cb8
00010000:00010000:1.0:1775764871.909422:0:72725:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f149153e400/0x23cab7dad48e60b7 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x7a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x929304787233021 expref: -99 pid: 72267 timeout: 0 lvb_type: 3 lru_score: 8 lru_type: 0
00010000:00010000:1.0:1775764871.909430:0:72725:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f149153e400/0x23cab7dad48e60b7 lrc: 2/0,0 mode: PR/PR res: [0x200000405:0x7a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x929304787233021 expref: -99 pid: 72267 timeout: 0 lvb_type: 3 lru_score: 9 lru_type: 0
00000080:00200000:1.0:1775764871.909479:0:72725:0:(vvp_io.c:1817:vvp_io_init()) [0x200000405:0x7a:0x0] ignore/verify layout 0/1, layout version 0 restore needed 0
00000001:00010000:1.0:1775764871.909492:0:72725:0:(glimpse.c:73:cl_glimpse_lock()) Glimpsing inode [0x200000405:0x7a:0x0]
00000008:00010000:1.0:1775764871.909523:0:72725:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: a
00000008:00010000:1.0:1775764871.909528:0:72725:0:(osc_lock.c:1310:osc_lock_init()) ### lock ffff9f14945e02b0, osc lock ffff9f148759a130, flags 0x400001002
00010000:00010000:1.0:1775764871.909547:0:72725:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-OST0001-osc-ffff9f14902b3800 lock: 000000002ccf2dff/0x23cab7dad48e60be lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0xb8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x92930478723302f expref: -99 pid: 72267 timeout: 0 lvb_type: 1 lru_score: 8 lru_type: 0
00010000:00010000:1.0:1775764871.909926:0:72725:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 18446744073709551615) ns: lustre-OST0001-osc-ffff9f14902b3800 lock: 000000002ccf2dff/0x23cab7dad48e60be lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0xb8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x92930478723302f expref: -99 pid: 72267 timeout: 0 lvb_type: 1 lru_score: 8 lru_type: 0
00000020:00200000:1.0:1775764871.909939:0:72725:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining.
00010000:00010000:1.0:1775764871.909942:0:72725:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-OST0001-osc-ffff9f14902b3800 lock: 000000002ccf2dff/0x23cab7dad48e60be lrc: 6/0,1 mode: PW/PW res: [0x2c0000401:0xb8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x810020000000000 nid: local remote: 0x92930478723302f expref: -99 pid: 72267 timeout: 0 lvb_type: 1 lru_score: 8 lru_type: 0
00010000:00010000:1.0:1775764871.909956:0:72725:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-OST0001-osc-ffff9f14902b3800 lock: 000000002ccf2dff/0x23cab7dad48e60be lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0xb8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x92930478723302f expref: -99 pid: 72267 timeout: 0 lvb_type: 1 lru_score: 9 lru_type: 1
00000020:00200000:1.0:1775764871.909969:0:72725:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining.
00000080:00200000:1.0:1775764871.909984:0:72725:0:(file.c:1742:ll_merge_attr_nolock()) [0x200000405:0x7a:0x0] updating i_size 1048576 i_blocks 0
00000020:00010000:1.0:1775764871.909990:0:72725:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff9f1486114490 (ffff9f1487c50128/0) at cl_lock_release():245
00000020:00010000:1.0:1775764871.909994:0:72725:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9f1486114490 (ffff9f1487c50128/0) at cl_lock_cancel():142
00000020:00010000:1.0:1775764871.909996:0:72725:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9f14945e02b0 (ffff9f1487c502d8/1) at cl_lock_cancel():142
00000020:00010000:1.0:1775764871.910000:0:72725:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9f1486114490 (ffff9f1487c50128/0) at cl_lock_fini():72
00000020:00010000:1.0:1775764871.910003:0:72725:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9f14945e02b0 (ffff9f1487c50128/1) at cl_lock_fini():72
00000080:00200000:1.0:1775764871.910116:0:72725:0:(vvp_io.c:288:vvp_io_fini()) [0x200000405:0x7a:0x0] ignore/verify layout 0/1, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0
00000080:00200000:1.0:1775764871.910253:0:72725:0:(file.c:6121:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9f1491deba88),name=/
00000002:00010000:1.0:1775764871.910268:0:72725:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0
00010000:00010000:1.0:1775764871.910277:0:72725:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f149817e000/0x23cab7dad48e61b3 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x9293047872332e4 expref: -99 pid: 72293 timeout: 0 lvb_type: 0 lru_score: 7 lru_type: 0
00010000:00010000:1.0:1775764871.910287:0:72725:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f149817e000/0x23cab7dad48e61b3 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x9293047872332e4 expref: -99 pid: 72293 timeout: 0 lvb_type: 0 lru_score: 7 lru_type: 0
00000080:00200000:1.0:1775764871.910294:0:72725:0:(namei.c:953:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1
00000080:00010000:1.0:1775764871.910300:0:72725:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9f1491deba88)
00000080:00010000:1.0:1775764871.910302:0:72725:0:(llite_internal.h:1984:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9f1491deba88) for lock 0x23cab7dad48e61b3
00000080:00010000:1.0:1775764871.910306:0:72725:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x23cab7dad48e61b3 from it ffffb06c01d47c10
00010000:00010000:1.0:1775764871.910309:0:72725:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f149817e000/0x23cab7dad48e61b3 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x9293047872332e4 expref: -99 pid: 72293 timeout: 0 lvb_type: 0 lru_score: 7 lru_type: 0
00010000:00010000:1.0:1775764871.910321:0:72725:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f149817e000/0x23cab7dad48e61b3 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x9293047872332e4 expref: -99 pid: 72293 timeout: 0 lvb_type: 0 lru_score: 8 lru_type: 0
00000080:00200000:1.0:1775764871.910330:0:72725:0:(file.c:6593:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9f1491deba88), inode mode 41ed mask 1
00000080:00200000:1.0:1775764871.910340:0:72725:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=f207.sanity-flr-tt, flags=0
00000080:00200000:1.0:1775764871.910344:0:72725:0:(file.c:6121:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9f1491deba88),name=/
00000002:00010000:1.0:1775764871.910351:0:72725:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0
00010000:00010000:1.0:1775764871.910357:0:72725:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f149817e000/0x23cab7dad48e61b3 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x9293047872332e4 expref: -99 pid: 72293 timeout: 0 lvb_type: 0 lru_score: 8 lru_type: 0
00010000:00010000:1.0:1775764871.910366:0:72725:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f149817e000/0x23cab7dad48e61b3 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x9293047872332e4 expref: -99 pid: 72293 timeout: 0 lvb_type: 0 lru_score: 8 lru_type: 0
00000080:00200000:1.0:1775764871.910372:0:72725:0:(namei.c:953:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1
00000080:00010000:1.0:1775764871.910377:0:72725:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9f1491deba88)
00000080:00010000:1.0:1775764871.910379:0:72725:0:(llite_internal.h:1984:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9f1491deba88) for lock 0x23cab7dad48e61b3
00000080:00010000:1.0:1775764871.910382:0:72725:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x23cab7dad48e61b3 from it ffffb06c01d47d68
00010000:00010000:1.0:1775764871.910385:0:72725:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f149817e000/0x23cab7dad48e61b3 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x9293047872332e4 expref: -99 pid: 72293 timeout: 0 lvb_type: 0 lru_score: 8 lru_type: 0
00010000:00010000:1.0:1775764871.910392:0:72725:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f149817e000/0x23cab7dad48e61b3 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x9293047872332e4 expref: -99 pid: 72293 timeout: 0 lvb_type: 0 lru_score: 9 lru_type: 0
00000080:00200000:1.0:1775764871.910400:0:72725:0:(file.c:6593:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9f1491deba88), inode mode 41ed mask 3
00000080:00200000:1.0:1775764871.910406:0:72725:0:(namei.c:2405:ll_unlink()) VFS Op:name=f207.sanity-flr-tt, dir=[0x200000007:0x1:0x0](ffff9f1491deba88)
00010000:00010000:1.0:1775764871.910451:0:72725:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f149817e000/0x23cab7dad48e61b3 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x9293047872332e4 expref: -99 pid: 72293 timeout: 0 lvb_type: 0 lru_score: 9 lru_type: 0
00010000:00010000:1.0:1775764871.910480:0:72725:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f149153e400/0x23cab7dad48e60b7 lrc: 2/0,0 mode: PR/PR res: [0x200000405:0x7a:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x929304787233021 expref: -99 pid: 72267 timeout: 0 lvb_type: 3 lru_score: 9 lru_type: 0
00000080:00200000:1.0:1775764871.910498:0:72725:0:(vvp_object.c:117:vvp_conf_set()) [0x200000405:0x7a:0x0]: losing layout lock
00010000:00010000:1.0:1775764871.910545:0:72725:0:(ldlm_request.c:1438:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f149153e400/0x23cab7dad48e60b7 lrc: 1/0,0 mode: --/PR res: [0x200000405:0x7a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x929304787233021 expref: -99 pid: 72267 timeout: 0 lvb_type: 3 lru_score: 9 lru_type: 0
00010000:00010000:1.0:1775764871.912676:0:72725:0:(ldlm_request.c:1438:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f149817e000/0x23cab7dad48e61b3 lrc: 1/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x9293047872332e4 expref: -99 pid: 72293 timeout: 0 lvb_type: 0 lru_score: 9 lru_type: 0
00010000:00010000:1.0:1775764871.912782:0:72725:0:(ldlm_request.c:1490:ldlm_cancel_pack()) 2 locks packed
00010000:00010000:1.0:1775764871.912788:0:72725:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f149153e400/0x23cab7dad48e60b7 lrc: 0/0,0 mode: --/PR res: [0x200000405:0x7a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x929304787233021 expref: -99 pid: 72267 timeout: 0 lvb_type: 3 lru_score: 9 lru_type: 0
00010000:00010000:1.0:1775764871.912813:0:72725:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f149817e000/0x23cab7dad48e61b3 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x9293047872332e4 expref: -99 pid: 72293 timeout: 0 lvb_type: 0 lru_score: 9 lru_type: 0
00000020:00100000:1.0:1775764871.912841:0:72725:0:(genops.c:2442:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9f14902b3800: modify RPC slot 1 is allocated opc 36, max 7
00000100:00100000:1.0:1775764871.912856:0:72725:0:(client.c:806:ptlrpc_reassign_next_xid()) @@@ reassign xid  req@ffff9f14ab4ff480 x1862022114176256/t0(0) o36->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 504/2128 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:1.0:1775764871.912890:0:72725:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff480 pname:cluuid:pid:xid:nid:opc:job rm:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:72725:1862022114176256:192.168.202.110@tcp:36:rm.0
00000100:00100000:1.0:1775764871.914046:0:72725:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f1488732b40 going to sleep for 50 seconds
00000080:00200000:1.0:1775764871.936319:0:72725:0:(llite_lib.c:2941:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 4096
00000080:00010000:1.0:1775764871.936329:0:72725:0:(llite_internal.h:1984:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9f1491deba88) for lock 0x23cab7dad48e620e
00000080:00010000:1.0:1775764871.936335:0:72725:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9f1491deba88)
00000080:00010000:1.0:1775764871.936338:0:72725:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x23cab7dad48e620e from it ffffb06c01d47b58
00010000:00010000:1.0:1775764871.936341:0:72725:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f149153fa00/0x23cab7dad48e620e lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x9293047872333e0 expref: -99 pid: 72725 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0
00010000:00010000:1.0:1775764871.936351:0:72725:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f149153fa00/0x23cab7dad48e620e lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x9293047872333e0 expref: -99 pid: 72725 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0
00000080:00200000:1.0:1775764871.936379:0:72725:0:(file.c:6593:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9f1491deba88), inode mode 41ed mask 1
00000080:00200000:1.0:1775764871.936395:0:72725:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=f207.sanity-flr, flags=8
00000080:00200008:1.0:1775764871.936410:0:72725:0:(file.c:6226:ll_getattr_dentry()) START file f207.sanity-flr:[0x200000405:0x7b:0x0](ffff9f14a1111148), request_mask 2047, flags 0, foreign 0
00000080:00200000:1.0:1775764871.936416:0:72725:0:(file.c:6121:ll_inode_revalidate()) VFS Op:inode=[0x200000405:0x7b:0x0](ffff9f14a1111148),name=f207.sanity-flr
00000002:00010000:1.0:1775764871.936438:0:72725:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: ,[0x200000405:0x7b:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0
00010000:00010000:1.0:1775764871.936448:0:72725:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f148628bc00/0x23cab7dad48e61c1 lrc: 2/1,0 mode: PR/PR res: [0x200000405:0x7b:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x92930478723332a expref: -99 pid: 72299 timeout: 0 lvb_type: 3 lru_score: 7 lru_type: 0
00010000:00010000:1.0:1775764871.936457:0:72725:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f148628bc00/0x23cab7dad48e61c1 lrc: 2/1,0 mode: PR/PR res: [0x200000405:0x7b:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x92930478723332a expref: -99 pid: 72299 timeout: 0 lvb_type: 3 lru_score: 7 lru_type: 0
00000080:00200000:1.0:1775764871.936464:0:72725:0:(namei.c:953:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1
00000080:00010000:1.0:1775764871.936469:0:72725:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000405:0x7b:0x0](ffff9f14a1111148)
00000080:00010000:1.0:1775764871.936471:0:72725:0:(llite_internal.h:1984:ll_set_lock_data()) setting l_data to inode [0x200000405:0x7b:0x0](ffff9f14a1111148) for lock 0x23cab7dad48e61c1
00000080:00010000:1.0:1775764871.936474:0:72725:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x23cab7dad48e61c1 from it ffffb06c01d47cb8
00010000:00010000:1.0:1775764871.936477:0:72725:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f148628bc00/0x23cab7dad48e61c1 lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x7b:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x92930478723332a expref: -99 pid: 72299 timeout: 0 lvb_type: 3 lru_score: 7 lru_type: 0
00010000:00010000:1.0:1775764871.936484:0:72725:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f148628bc00/0x23cab7dad48e61c1 lrc: 2/0,0 mode: PR/PR res: [0x200000405:0x7b:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x92930478723332a expref: -99 pid: 72299 timeout: 0 lvb_type: 3 lru_score: 8 lru_type: 0
00000080:00200000:1.0:1775764871.936521:0:72725:0:(file.c:6121:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9f1491deba88),name=/
00000002:00010000:1.0:1775764871.936529:0:72725:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0
00010000:00010000:1.0:1775764871.936534:0:72725:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f149153fa00/0x23cab7dad48e620e lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x9293047872333e0 expref: -99 pid: 72725 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0
00010000:00010000:1.0:1775764871.936542:0:72725:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f149153fa00/0x23cab7dad48e620e lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x9293047872333e0 expref: -99 pid: 72725 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0
00000080:00200000:1.0:1775764871.936549:0:72725:0:(namei.c:953:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1
00000080:00010000:1.0:1775764871.936588:0:72725:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9f1491deba88)
00000080:00010000:1.0:1775764871.936592:0:72725:0:(llite_internal.h:1984:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9f1491deba88) for lock 0x23cab7dad48e620e
00000080:00010000:1.0:1775764871.936604:0:72725:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x23cab7dad48e620e from it ffffb06c01d47c10
00010000:00010000:1.0:1775764871.936607:0:72725:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f149153fa00/0x23cab7dad48e620e lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x9293047872333e0 expref: -99 pid: 72725 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0
00010000:00010000:1.0:1775764871.936614:0:72725:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f149153fa00/0x23cab7dad48e620e lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x9293047872333e0 expref: -99 pid: 72725 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0
00000080:00200000:1.0:1775764871.936622:0:72725:0:(file.c:6593:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9f1491deba88), inode mode 41ed mask 1
00000080:00200000:1.0:1775764871.936630:0:72725:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=f207.sanity-flr, flags=0
00000080:00200000:1.0:1775764871.936634:0:72725:0:(file.c:6121:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9f1491deba88),name=/
00000002:00010000:1.0:1775764871.936641:0:72725:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0
00010000:00010000:1.0:1775764871.936647:0:72725:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f149153fa00/0x23cab7dad48e620e lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x9293047872333e0 expref: -99 pid: 72725 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0
00010000:00010000:1.0:1775764871.936655:0:72725:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f149153fa00/0x23cab7dad48e620e lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x9293047872333e0 expref: -99 pid: 72725 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0
00000080:00200000:1.0:1775764871.936660:0:72725:0:(namei.c:953:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1
00000080:00010000:1.0:1775764871.936664:0:72725:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9f1491deba88)
00000080:00010000:1.0:1775764871.936667:0:72725:0:(llite_internal.h:1984:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9f1491deba88) for lock 0x23cab7dad48e620e
00000080:00010000:1.0:1775764871.936670:0:72725:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x23cab7dad48e620e from it ffffb06c01d47d68
00010000:00010000:1.0:1775764871.936672:0:72725:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f149153fa00/0x23cab7dad48e620e lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x9293047872333e0 expref: -99 pid: 72725 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0
00010000:00010000:1.0:1775764871.936678:0:72725:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f149153fa00/0x23cab7dad48e620e lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x9293047872333e0 expref: -99 pid: 72725 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0
00000080:00200000:1.0:1775764871.936700:0:72725:0:(file.c:6593:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9f1491deba88), inode mode 41ed mask 3
00000080:00200000:1.0:1775764871.936707:0:72725:0:(namei.c:2405:ll_unlink()) VFS Op:name=f207.sanity-flr, dir=[0x200000007:0x1:0x0](ffff9f1491deba88)
00010000:00010000:1.0:1775764871.936778:0:72725:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f149153fa00/0x23cab7dad48e620e lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x9293047872333e0 expref: -99 pid: 72725 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0
00010000:00010000:1.0:1775764871.936802:0:72725:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f148628bc00/0x23cab7dad48e61c1 lrc: 2/0,0 mode: PR/PR res: [0x200000405:0x7b:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x92930478723332a expref: -99 pid: 72299 timeout: 0 lvb_type: 3 lru_score: 8 lru_type: 0
00000080:00200000:1.0:1775764871.937249:0:72725:0:(vvp_object.c:117:vvp_conf_set()) [0x200000405:0x7b:0x0]: losing layout lock
00010000:00010000:1.0:1775764871.937273:0:72725:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f148628a000/0x23cab7dad48e61c8 lrc: 2/0,0 mode: CR/CR res: [0x200000405:0x7b:0x0].0x0 bits 0xd/0x0 rrc: 5 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x929304787233338 expref: -99 pid: 72299 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0
00000002:00100000:1.0:1775764871.937330:0:72725:0:(mdc_request.c:964:mdc_close()) @@@ matched open  req@ffff9f14ab4b7b80 x1862022114172416/t8589936682(8589936682) o101->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 936/1024 e 0 to 0 dl 1775764919 ref 1 fl Complete:RPQU/604/ffffffff rc 0/-1 job:'lfs.0' uid:0 gid:0 projid:0
00000020:00100000:1.0:1775764871.937342:0:72725:0:(genops.c:2442:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9f14902b3800: modify RPC slot 1 is allocated opc 35, max 7
00000100:00100000:1.0:1775764871.937347:0:72725:0:(client.c:806:ptlrpc_reassign_next_xid()) @@@ reassign xid  req@ffff9f14ab4fea00 x1862022114176640/t0(0) o35->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:23/10 lens 392/2128 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0
00000100:00100000:1.0:1775764871.937374:0:72725:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job rm:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:72725:1862022114176640:192.168.202.110@tcp:35:rm.0
00000100:00100000:1.0:1775764871.937483:0:72725:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f1488732b40 going to sleep for 11 seconds
00010000:00010000:1.0:1775764875.530894:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775764877.845821:0:2404:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1]
00000100:00100000:1.0:1775764877.845833:0:2404:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2404:1862022114177920:192.168.202.110@tcp:400:kworker.0
00000100:00100000:1.0:1775764877.845926:0:2404:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ffb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2404:1862022114178048:192.168.202.110@tcp:400:kworker.0
00000100:00100000:1.0:1775764877.845996:0:2404:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1]
00000100:00100000:1.0:1775764877.846003:0:2404:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2404:1862022114178176:192.168.202.110@tcp:400:kworker.0
00000100:00100000:1.0:1775764878.064179:0:2396:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.110@tcp (source: 12345-192.168.202.110@tcp)
00000100:00100000:1.0:1775764878.064229:0:2396:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.110@tcp (source: 12345-192.168.202.110@tcp)
00000100:00100000:1.0:1775764878.064245:0:2396:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.110@tcp (source: 12345-192.168.202.110@tcp)
00000100:00100000:1.0:1775764878.064264:0:2396:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.110@tcp (source: 12345-192.168.202.110@tcp)
00000100:00100000:1.0:1775764878.064281:0:2396:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.110@tcp (source: 12345-192.168.202.110@tcp)
00000100:00100000:1.0:1775764878.064322:0:2396:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.110@tcp (source: 12345-192.168.202.110@tcp)
00000100:00100000:1.0:1775764882.920833:0:73557:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4b6300 pname:cluuid:pid:xid:nid:opc:job mount.lustre:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:73557:1862022114180096:192.168.202.110@tcp:101:mount.lustre.0
00010000:00010000:1.0:1775764882.920860:0:73557:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGC192.168.202.110@tcp lock: ffff9f148628b800/0x23cab7dad48e624d lrc: 4/1,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 3 type: PLN flags: 0x11000000000000 nid: local remote: 0x929304787233442 expref: -99 pid: 73557 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0
00010000:00010000:1.0:1775764882.920874:0:73557:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: MGC192.168.202.110@tcp lock: ffff9f148628b800/0x23cab7dad48e624d lrc: 4/1,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 3 type: PLN flags: 0x1000000000000 nid: local remote: 0x929304787233442 expref: -99 pid: 73557 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0
00000100:00100000:1.0:1775764882.920952:0:73557:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job mount.lustre:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:73557:1862022114180224:192.168.202.110@tcp:501:mount.lustre.0
00000100:00100000:1.0:1775764882.921032:0:73557:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14912a1600 going to sleep for 11 seconds
00000100:00100000:1.0:1775764882.926681:0:73557:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job mount.lustre:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:73557:1862022114180224:192.168.202.110@tcp:501:mount.lustre.0
00000100:00100000:1.0:1775764882.926754:0:73557:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job mount.lustre:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:73557:1862022114180352:192.168.202.110@tcp:503:mount.lustre.0
00000100:00100000:1.0:1775764882.926875:0:73557:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14912a1600 going to sleep for 11 seconds
00000100:00100000:1.0:1775764882.938254:0:73557:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job mount.lustre:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:73557:1862022114180352:192.168.202.110@tcp:503:mount.lustre.0
00000100:00100000:1.0:1775764882.951402:0:73571:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job llog_process_th:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:73571:1862022114180480:192.168.202.110@tcp:502:llog_process_th.0
00000100:00100000:1.0:1775764882.951637:0:73571:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14912a1600 going to sleep for 11 seconds
00000100:00100000:1.0:1775764882.973501:0:73571:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job llog_process_th:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:73571:1862022114180480:192.168.202.110@tcp:502:llog_process_th.0
00000020:01000000:1.0:1775764882.973551:0:73571:0:(obd_config.c:1888:class_config_llog_handler()) Marker, inst_flg=0x0 mark_flg=0x1
00000020:00000080:1.0:1775764882.973589:0:73571:0:(obd_config.c:1468:class_process_config()) processing cmd: cf010
00000020:00000080:1.0:1775764882.973592:0:73571:0:(obd_config.c:1559:class_process_config()) marker 2 (0x1) general debug_raw_pointers
00000020:01000000:1.0:1775764882.973599:0:73571:0:(obd_config.c:2003:class_config_llog_handler()) cmd ce032, instance name: general-ffff9f14b38c6800
00000020:00000080:1.0:1775764882.973602:0:73571:0:(obd_config.c:1468:class_process_config()) processing cmd: ce032
00000020:01000000:1.0:1775764882.973610:0:73571:0:(obd_config.c:1888:class_config_llog_handler()) Marker, inst_flg=0x2 mark_flg=0x2
00000020:00000080:1.0:1775764882.973613:0:73571:0:(obd_config.c:1468:class_process_config()) processing cmd: cf010
00000020:00000080:1.0:1775764882.973614:0:73571:0:(obd_config.c:1559:class_process_config()) marker 2 (0x2) general debug_raw_pointers
00000020:01000000:1.0:1775764882.973617:0:73571:0:(obd_config.c:1888:class_config_llog_handler()) Marker, inst_flg=0x0 mark_flg=0x1
00000020:00000080:1.0:1775764882.973619:0:73571:0:(obd_config.c:1468:class_process_config()) processing cmd: cf010
00000020:00000080:1.0:1775764882.973621:0:73571:0:(obd_config.c:1559:class_process_config()) marker 3 (0x1) general lod.*.mdt_hash
00000020:01000000:1.0:1775764882.973624:0:73571:0:(obd_config.c:2003:class_config_llog_handler()) cmd ce032, instance name: general-ffff9f14b38c6800
00000020:00000080:1.0:1775764882.973626:0:73571:0:(obd_config.c:1468:class_process_config()) processing cmd: ce032
00000100:00100000:1.0:1775764882.973719:0:73571:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job llog_process_th:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:73571:1862022114181120:192.168.202.110@tcp:502:llog_process_th.0
00000100:00100000:1.0:1775764882.973810:0:73571:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14912a1600 going to sleep for 11 seconds
00000100:00100000:1.0:1775764882.985021:0:73571:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job llog_process_th:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:73571:1862022114181120:192.168.202.110@tcp:502:llog_process_th.0
00000020:01000000:1.0:1775764882.985094:0:73571:0:(obd_config.c:1888:class_config_llog_handler()) Marker, inst_flg=0x2 mark_flg=0x2
00000020:00000080:1.0:1775764882.985107:0:73571:0:(obd_config.c:1468:class_process_config()) processing cmd: cf010
00000020:00000080:1.0:1775764882.985109:0:73571:0:(obd_config.c:1559:class_process_config()) marker 3 (0x2) general lod.*.mdt_hash
00000040:00080000:1.0:1775764882.985116:0:73571:0:(llog.c:857:llog_process_thread()) stop processing plain [0xa:0x2:0x0] index 7 count 7
00000100:00100000:1.0:1775764882.985647:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114181248:192.168.202.110@tcp:38:mount.lustre.0
00000100:00100000:1.0:1775764882.985866:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114181504:192.168.202.110@tcp:38:mount.lustre.0
00000100:00100000:1.0:1775764882.998617:0:2403:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:f7291b10-c63d-4c01-a1d1-66463b7d0b42:0:1862022114181376:192.168.202.110@tcp:41:mount.lustre.0
00000100:00100000:1.0:1775764883.006998:0:2403:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:f7291b10-c63d-4c01-a1d1-66463b7d0b42:0:1862022114181632:192.168.202.110@tcp:41:mount.lustre.0
00000100:00100000:1.0:1775764883.022055:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4b7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114182144:192.168.202.110@tcp:8:mount.lustre.0
00000100:00100000:1.0:1775764883.025759:0:2402:0:(client.c:3082:ptlrpc_free_committed()) lustre-OST0001-osc-ffff9f14b38c6800: committing for last_committed 0 gen 1
00000100:00080000:1.0:1775764883.025776:0:2402:0:(import.c:1130:ptlrpc_connect_interpret()) lustre-OST0001-osc-ffff9f14b38c6800: connect to target with instance 4
00000100:00080000:1.0:1775764883.025787:0:2402:0:(import.c:958:ptlrpc_connect_set_flags()) lustre-OST0001-osc-ffff9f14b38c6800: Resetting ns_connect_flags to server flags: 0xa0425af2e3440038
00000080:00000004:1.0:1775764883.025800:0:2402:0:(lcommon_misc.c:94:cl_ocd_update()) Changing connect_flags: 0xa0465af2e3640038 -> 0xa0425af2e3440038
00000080:00080000:1.0:1775764883.025806:0:2402:0:(lcommon_misc.c:56:cl_init_ea_size()) updating def/max_easize: 72/65536
00000100:00080000:1.0:1775764883.025816:0:2402:0:(import.c:1211:ptlrpc_connect_interpret()) connected to replayable target: lustre-OST0001_UUID
00000100:00080000:1.0:1775764883.025821:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618800 lustre-OST0001_UUID: changing import state from CONNECTING to FULL
00020000:01000000:1.0:1775764883.025829:0:2402:0:(lov_obd.c:392:lov_set_osc_active()) Marking OSC lustre-OST0001_UUID active
00000080:00000004:1.0:1775764883.025832:0:2402:0:(lcommon_misc.c:94:cl_ocd_update()) Changing connect_flags: 0xa0425af2e3440038 -> 0xa0425af2e3440038
00000080:00080000:1.0:1775764883.025834:0:2402:0:(lcommon_misc.c:56:cl_init_ea_size()) updating def/max_easize: 72/65536
00000100:00100000:1.0:1775764883.025846:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4b7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114182144:192.168.202.110@tcp:8:mount.lustre.0
00010000:00010000:1.0:1775764885.771503:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000080:00200000:1.0:1775764887.797504:0:73694:0:(file.c:6121:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9f1491deba88),name=/
00000002:00010000:1.0:1775764887.797535:0:73694:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0
00010000:00010000:1.0:1775764887.797783:0:73694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f14a04c5e00/0x23cab7dad48e627e lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 73694 initiator: MDT0 lru_score: 0 lru_type: 0
00010000:00010000:1.0:1775764887.797794:0:73694:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f14a04c5e00/0x23cab7dad48e627e lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 73694 initiator: MDT0 lru_score: 0 lru_type: 0
00010000:00010000:1.0:1775764887.797806:0:73694:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f14a04c5e00/0x23cab7dad48e627e lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 73694 initiator: MDT0 lru_score: 0 lru_type: 0
00000100:00100000:1.0:1775764887.797838:0:73694:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job lfs:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:73694:1862022114182528:192.168.202.110@tcp:101:lfs.0
00000100:00100000:1.0:1775764887.798061:0:73694:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14912a1180 going to sleep for 50 seconds
00000100:00100000:1.0:1775764887.810116:0:2395:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.110@tcp (source: 12345-192.168.202.110@tcp)
00000080:00200000:1.0:1775764887.832101:0:73374:0:(file.c:6121:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9f1491deba88),name=/
00000002:00010000:1.0:1775764887.832126:0:73374:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0
00010000:00010000:1.0:1775764887.832194:0:73374:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f14a04c5600/0x23cab7dad48e629a lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 73374 initiator: MDT0 lru_score: 0 lru_type: 0
00010000:00010000:1.0:1775764887.833079:0:73374:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f14a04c5600/0x23cab7dad48e629a lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 73374 initiator: MDT0 lru_score: 0 lru_type: 0
00010000:00010000:1.0:1775764887.833093:0:73374:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f14a04c5600/0x23cab7dad48e629a lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 73374 initiator: MDT0 lru_score: 0 lru_type: 0
00000100:00100000:1.0:1775764887.833133:0:73374:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job bash:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:73374:1862022114183936:192.168.202.110@tcp:101:bash.0
00000100:00100000:1.0:1775764887.833323:0:73374:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14912a1600 going to sleep for 50 seconds
00000080:00200000:1.0:1775764887.889612:0:73695:0:(file.c:6121:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9f1491deba88),name=/
00000002:00010000:1.0:1775764887.889642:0:73695:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0
00010000:00010000:1.0:1775764887.889720:0:73695:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f14a04c5600/0x23cab7dad48e62b6 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 73695 initiator: MDT0 lru_score: 0 lru_type: 0
00010000:00010000:1.0:1775764887.889731:0:73695:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f14a04c5600/0x23cab7dad48e62b6 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 73695 initiator: MDT0 lru_score: 0 lru_type: 0
00010000:00010000:1.0:1775764887.889742:0:73695:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f14a04c5600/0x23cab7dad48e62b6 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 73695 initiator: MDT0 lru_score: 0 lru_type: 0
00000100:00100000:1.0:1775764887.889768:0:73695:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff480 pname:cluuid:pid:xid:nid:opc:job lfs:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:73695:1862022114185216:192.168.202.110@tcp:101:lfs.0
00000100:00100000:1.0:1775764887.889868:0:73695:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14912a1600 going to sleep for 50 seconds
00000100:00100000:1.0:1775764887.912072:0:2395:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.110@tcp (source: 12345-192.168.202.110@tcp)
00000100:00100000:1.0:1775764887.912124:0:39304:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1862022124557184
00000100:00100000:1.0:1775764887.912135:0:39304:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1862022124557184
00000100:00100000:1.0:1775764887.912185:0:39304:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.110@tcp, seq: 6443
00000100:00100000:1.0:1775764887.912194:0:39304:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9f14ab4fd880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48+5:6484:x1862022124557184:12345-192.168.202.110@tcp:104:
00010000:00010000:1.0:1775764887.912211:0:39304:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast  ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f148628a600/0x23cab7dad48e62c4 lrc: 2/0,0 mode: PR/PR res: [0x200000405:0x80:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x420000000000 nid: local remote: 0x929304787233537 expref: -99 pid: 73695 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0
00010000:00010000:1.0:1775764887.912297:0:39304:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool
00000100:00100000:1.0:1775764887.912303:0:39304:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9f14ab4fd880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48+5:6484:x1862022124557184:12345-192.168.202.110@tcp:104: Request processed in 111us (240us total) trans 0 rc 0/0
00000100:00100000:1.0:1775764887.912311:0:39304:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.110@tcp, seq: 6443
00000100:00100000:1.0:1775764887.968375:0:39305:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1862022124557312
00000100:00100000:1.0:1775764887.968385:0:39305:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1862022124557312
00000100:00100000:1.0:1775764887.968481:0:39305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.110@tcp, seq: 6444
00000100:00100000:1.0:1775764887.968551:0:39305:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9f14a04d3800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:LOV_OSC_UUID+4:27790:x1862022124557312:12345-192.168.202.110@tcp:104:
00010000:00010000:1.0:1775764887.968608:0:39305:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast  ns: lustre-OST0000-osc-ffff9f14902b3800 lock: 00000000f4e9b72d/0x23cab7dad48e62af lrc: 3/0,0 mode: PW/PW res: [0x280000401:0xbb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800420080000000 nid: local remote: 0x9293047872334f8 expref: -99 pid: 73374 timeout: 0 lvb_type: 1 lru_score: 5 lru_type: 0
00010000:00010000:1.0:1775764887.968680:0:39305:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool
00000100:00100000:1.0:1775764887.968687:0:39305:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9f14a04d3800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:LOV_OSC_UUID+4:27790:x1862022124557312:12345-192.168.202.110@tcp:104: Request processed in 199us (458us total) trans 0 rc 0/0
00000100:00100000:1.0:1775764887.968704:0:39305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.110@tcp, seq: 6444
00000100:00100000:1.0:1775764888.039518:0:73695:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4b6680 pname:cluuid:pid:xid:nid:opc:job lfs:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:73695:1862022114187776:192.168.202.110@tcp:101:lfs.0
00010000:00010000:1.0:1775764888.039548:0:73695:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f148628b400/0x23cab7dad48e62ee lrc: 4/1,0 mode: --/PR res: [0x200000405:0x80:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x9293047872335a0 expref: -99 pid: 73695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0
00010000:00010000:1.0:1775764888.039595:0:73695:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f148628b400/0x23cab7dad48e62ee lrc: 4/1,0 mode: PR/PR res: [0x200000405:0x80:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x9293047872335a0 expref: -99 pid: 73695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0
00010000:00010000:1.0:1775764888.039607:0:73695:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f148628b400/0x23cab7dad48e62ee lrc: 4/1,0 mode: PR/PR res: [0x200000405:0x80:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x9293047872335a0 expref: -99 pid: 73695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0
00000002:00100000:1.0:1775764888.039622:0:73695:0:(mdc_locks.c:833:mdc_finish_enqueue()) @@@ op=80 disposition=0, status=0  req@ffff9f14ab4b6680 x1862022114187776/t0(0) o101->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 568/1152 e 0 to 0 dl 1775764943 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'lfs.0' uid:0 gid:0 projid:0
00000002:00010000:1.0:1775764888.039634:0:73695:0:(mdc_locks.c:1357:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f148628b400/0x23cab7dad48e62ee lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x80:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x9293047872335a0 expref: -99 pid: 73695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0
00010000:00010000:1.0:1775764888.039647:0:73695:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9f1494571c00 type 13 mode 4 res 8589935621/128 (0 0)
00000080:00200000:1.0:1775764888.039654:0:73695:0:(namei.c:953:ll_intent_lock()) intent lock 128 on i1 [0x200000405:0x80:0x0] suppgids 0 -1: rc 0
00000080:00010000:1.0:1775764888.039681:0:73695:0:(llite_internal.h:1984:ll_set_lock_data()) setting l_data to inode [0x200000405:0x80:0x0](ffff9f1491d300c8) for lock 0x23cab7dad48e62ee
00000080:00010000:1.0:1775764888.039688:0:73695:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x23cab7dad48e62ee from it ffffb06c023678f8
00010000:00010000:1.0:1775764888.039693:0:73695:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f148628b400/0x23cab7dad48e62ee lrc: 3/1,0 mode: PR/PR res: [0x200000405:0x80:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x9293047872335a0 expref: -99 pid: 73695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0
00010000:00010000:1.0:1775764888.039703:0:73695:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f148628b400/0x23cab7dad48e62ee lrc: 2/0,0 mode: PR/PR res: [0x200000405:0x80:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x9293047872335a0 expref: -99 pid: 73695 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0
00000080:00200000:1.0:1775764888.039750:0:73695:0:(rw26.c:395:ll_direct_IO()) VFS Op:inode=[0x200000405:0x80:0x0](ffff9f1491d300c8), size=5 (max 536870912), offset=0=0x0, pages 1 (max 131072), lockless, parallel, unaligned
00000020:00200000:1.0:1775764888.039811:0:73695:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining.
00000080:00200000:1.0:1775764888.039827:0:73695:0:(rw26.c:531:ll_direct_IO()) result 0 tot_bytes 5 count 5 file_offset 5
00000080:00200000:1.0:1775764888.039844:0:73695:0:(file.c:1742:ll_merge_attr_nolock()) [0x200000405:0x80:0x0] updating i_size 0 i_blocks 0
00000020:00010000:1.0:1775764888.039850:0:73695:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff9f14881dd308 (ffff9f1487c3d3f8/0) at cl_lock_release():245
00000020:00010000:1.0:1775764888.039853:0:73695:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9f14881dd308 (ffff9f1487c3d3f8/0) at cl_lock_cancel():142
00000020:00010000:1.0:1775764888.039857:0:73695:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9f149457db30 (ffff9f1487c3dea8/1) at cl_lock_cancel():142
00000020:00010000:1.0:1775764888.039861:0:73695:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9f14881dd308 (ffff9f1487c3d3f8/0) at cl_lock_fini():72
00000020:00010000:1.0:1775764888.039863:0:73695:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9f149457db30 (ffff9f1487c3d3f8/1) at cl_lock_fini():72
00000080:00200000:1.0:1775764888.039871:0:73695:0:(vvp_io.c:517:vvp_io_advance()) advancing 5 bytes
00000020:00200000:1.0:1775764888.039877:0:73695:0:(cl_io.c:1822:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining.
00000008:00010000:1.0:1775764888.039921:0:70315:0:(osc_cache.c:2012:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff9f14a04c9540
00000020:00200000:1.0:1775764888.039932:0:70315:0:(cl_io.c:1678:__ll_dio_user_copy()) count 5, offset 0, pos 0, cdp_page_count 1
00000008:00100000:1.0:1775764888.040012:0:70315:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff9f14ab4fed80 - object 0x2c0000401:188 offset 0<>5
00000100:00100000:1.0:1775764888.047218:0:44145:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1862022124557440
00000100:00100000:1.0:1775764888.047228:0:44145:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1862022124557440
00000100:00100000:1.0:1775764888.047328:0:44145:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.110@tcp, seq: 6445
00000100:00100000:1.0:1775764888.047341:0:44145:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9f14a04d0000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:LOV_OSC_UUID+4:23978:x1862022124557440:12345-192.168.202.110@tcp:104:
00010000:00010000:1.0:1775764888.047362:0:44145:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast  ns: lustre-OST0001-osc-ffff9f14902b3800 lock: 0000000082832049/0x23cab7dad48e62d9 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0xbc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800420000000000 nid: local remote: 0x929304787233568 expref: -99 pid: 73695 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0
00010000:00010000:1.0:1775764888.047456:0:44145:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool
00000100:00100000:1.0:1775764888.047463:0:44145:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9f14a04d0000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:LOV_OSC_UUID+4:23978:x1862022124557440:12345-192.168.202.110@tcp:104: Request processed in 126us (576us total) trans 0 rc 0/0
00000100:00100000:1.0:1775764888.047473:0:44145:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.110@tcp, seq: 6445
00000100:00100000:1.0:1775764888.180004:0:39305:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1862022124557568
00000100:00100000:1.0:1775764888.180091:0:39305:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1862022124557568
00000100:00100000:1.0:1775764888.180194:0:39305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.110@tcp, seq: 6446
00000100:00100000:1.0:1775764888.180205:0:39305:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9f14a04d0380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48+5:7675:x1862022124557568:12345-192.168.202.110@tcp:104:
00010000:00010000:1.0:1775764888.180224:0:39305:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast  ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f148628b400/0x23cab7dad48e62ee lrc: 2/0,0 mode: PR/PR res: [0x200000405:0x80:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x9293047872335a0 expref: -99 pid: 73695 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0
00010000:00010000:1.0:1775764888.180324:0:39305:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool
00000100:00100000:1.0:1775764888.180331:0:39305:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9f14a04d0380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48+5:7675:x1862022124557568:12345-192.168.202.110@tcp:104: Request processed in 129us (817us total) trans 0 rc 0/0
00000100:00100000:1.0:1775764888.180338:0:39305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.110@tcp, seq: 6446
00000100:00100000:1.0:1775764888.198788:0:44145:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1862022124557696
00000100:00100000:1.0:1775764888.198799:0:44145:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1862022124557696
00000100:00100000:1.0:1775764888.198859:0:44145:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.110@tcp, seq: 6447
00000100:00100000:1.0:1775764888.198879:0:44145:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9f14a04d0e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:LOV_OSC_UUID+4:28868:x1862022124557696:12345-192.168.202.110@tcp:104:
00010000:00010000:1.0:1775764888.198907:0:44145:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast  ns: lustre-OST0000-osc-ffff9f14902b3800 lock: 0000000046c11ee4/0x23cab7dad48e62e7 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0xbb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800420000000000 nid: local remote: 0x92930478723358b expref: -99 pid: 73695 timeout: 0 lvb_type: 1 lru_score: 4 lru_type: 0
00010000:00010000:1.0:1775764888.199005:0:44145:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool
00000100:00100000:1.0:1775764888.199779:0:44145:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9f14a04d0e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:LOV_OSC_UUID+4:28868:x1862022124557696:12345-192.168.202.110@tcp:104: Request processed in 910us (1100us total) trans 0 rc 0/0
00000100:00100000:1.0:1775764888.199788:0:44145:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.110@tcp, seq: 6447
00000100:00100000:1.0:1775764888.205640:0:39305:0:(service.c:153:ptlrpc_grow_req_bufs()) ldlm_cbd: allocate 1 new 8192-byte reqbufs (0/2 left), rc = 0
00000100:00100000:1.0:1775764888.205660:0:39305:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1862022124557824
00000100:00100000:1.0:1775764888.205669:0:39305:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1862022124557824
00000100:00100000:1.0:1775764888.205733:0:39305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.110@tcp, seq: 6448
00000100:00100000:1.0:1775764888.205743:0:39305:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9f1484cfb450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:LOV_OSC_UUID+4:6471:x1862022124557824:12345-192.168.202.110@tcp:105:
00010000:00010000:1.0:1775764888.205759:0:39305:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast  ns: lustre-OST0000-osc-ffff9f14902b3800 lock: 0000000053880a67/0x23cab7dad48e6303 lrc: 4/0,1 mode: --/PW res: [0x280000401:0xbb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x0 nid: local remote: 0x9293047872335d8 expref: -99 pid: 73695 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0
00010000:00010000:1.0:1775764888.205773:0:39305:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-OST0000-osc-ffff9f14902b3800 lock: 0000000053880a67/0x23cab7dad48e6303 lrc: 4/0,1 mode: --/PW res: [0x280000401:0xbb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x0 nid: local remote: 0x9293047872335d8 expref: -99 pid: 73695 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0
00010000:00010000:1.0:1775764888.205782:0:39305:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-OST0000-osc-ffff9f14902b3800 lock: 0000000053880a67/0x23cab7dad48e6303 lrc: 4/0,1 mode: --/PW res: [0x280000401:0xbb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10000000000000 nid: local remote: 0x9293047872335d8 expref: -99 pid: 73695 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0
00010000:00010000:1.0:1775764888.205798:0:39305:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-OST0000-osc-ffff9f14902b3800 lock: 0000000053880a67/0x23cab7dad48e6303 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0xbb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10080000000000 nid: local remote: 0x9293047872335d8 expref: -99 pid: 73695 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0
00010000:00010000:1.0:1775764888.205807:0:39305:0:(ldlm_resource.c:1714:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-OST0000-osc-ffff9f14902b3800 lock: 0000000053880a67/0x23cab7dad48e6303 lrc: 5/0,1 mode: PW/PW res: [0x280000401:0xbb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10080000000000 nid: local remote: 0x9293047872335d8 expref: -99 pid: 73695 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0
00010000:00010000:1.0:1775764888.205818:0:39305:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-OST0000-osc-ffff9f14902b3800 lock: 0000000053880a67/0x23cab7dad48e6303 lrc: 5/0,1 mode: PW/PW res: [0x280000401:0xbb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x80000000000 nid: local remote: 0x9293047872335d8 expref: -99 pid: 73695 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0
00010000:00010000:1.0:1775764888.205835:0:39305:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffff9f148628be00)
00000100:00100000:1.0:1775764888.205997:0:39305:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9f1484cfb450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:LOV_OSC_UUID+4:6471:x1862022124557824:12345-192.168.202.110@tcp:105: Request processed in 255us (1003us total) trans 0 rc 0/0
00000100:00100000:1.0:1775764888.206004:0:39305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.110@tcp, seq: 6448
00000100:00100000:1.0:1775764888.231295:0:2395:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.110@tcp (source: 12345-192.168.202.110@tcp)
00000100:00100000:1.0:1775764888.231351:0:2395:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.110@tcp (source: 12345-192.168.202.110@tcp)
00000100:00100000:1.0:1775764888.231374:0:39305:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1862022124557952
00000100:00100000:1.0:1775764888.231381:0:39305:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1862022124557952
00000100:00100000:1.0:1775764888.231414:0:39305:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1862022124558080
00000100:00100000:1.0:1775764888.231415:0:39305:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1862022124558080
00000100:00100000:1.0:1775764888.231424:0:39305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.110@tcp, seq: 6449
00000100:00100000:1.0:1775764888.231430:0:39305:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9f14ab4fdf80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48+6:6487:x1862022124557952:12345-192.168.202.110@tcp:104:
00010000:00010000:1.0:1775764888.231455:0:39305:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast  ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f149153e400/0x23cab7dad48e62a8 lrc: 2/0,0 mode: PR/PR res: [0x200000405:0x7e:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x9293047872334ea expref: -99 pid: 73374 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0
00010000:00010000:1.0:1775764888.232484:0:39305:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to priority blp list, 2 blwis in pool
00000100:00100000:1.0:1775764888.232492:0:39305:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9f14ab4fdf80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48+6:6487:x1862022124557952:12345-192.168.202.110@tcp:104: Request processed in 1061us (1208us total) trans 0 rc 0/0
00000100:00100000:1.0:1775764888.232499:0:39305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.110@tcp, seq: 6449
00000100:00100000:1.0:1775764888.244003:0:39305:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1862022124558208
00000100:00100000:1.0:1775764888.244013:0:39305:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1862022124558208
00000100:00100000:1.0:1775764888.244077:0:39305:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.110@tcp, seq: 6451
00000100:00100000:1.0:1775764888.244087:0:39305:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9f14a04d2680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48+5:6487:x1862022124558208:12345-192.168.202.110@tcp:104:
00010000:00010000:1.0:1775764888.244104:0:39305:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast  ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f148628b000/0x23cab7dad48e62f5 lrc: 2/0,0 mode: CR/CR res: [0x200000405:0x80:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x420000000000 nid: local remote: 0x9293047872335bc expref: -99 pid: 73695 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0
00010000:00010000:1.0:1775764888.244185:0:39305:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool
00000100:00100000:1.0:1775764888.244192:0:39305:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9f14a04d2680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48+5:6487:x1862022124558208:12345-192.168.202.110@tcp:104: Request processed in 107us (368us total) trans 0 rc 0/0
00000100:00100000:1.0:1775764888.244200:0:39305:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.110@tcp, seq: 6451
00010000:00010000:1.0:1775764888.843736:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00010000:00010000:1.0:1775764890.050814:0:73828:0:(ldlm_resource.c:195:lru_size_store()) dropping all unused locks from namespace lustre-MDT0000-mdc-ffff9f14b38c6800
00010000:00010000:1.0:1775764890.050855:0:73828:0:(ldlm_resource.c:195:lru_size_store()) dropping all unused locks from namespace lustre-MDT0001-mdc-ffff9f14902b3800
00010000:00010000:1.0:1775764890.050876:0:73828:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 4/4 locks to regular blp list, 1 blwis in pool
00010000:00010000:1.0:1775764890.050886:0:60134:0:(ldlm_lockd.c:2794:ldlm_bl_get_work()) Got 4 locks of 0 total in blp.  (0 blwis in pool)
00010000:00010000:1.0:1775764890.050894:0:60134:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9f14902b3800 lock: ffff9f14c2152400/0x23cab7dad48e5baf lrc: 2/0,0 mode: PR/PR res: [0x240000406:0xb:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x929304787231af7 expref: -99 pid: 68370 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0
00010000:00010000:1.0:1775764890.050918:0:60134:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9f14902b3800 lock: ffff9f149817ee00/0x23cab7dad48e5a7b lrc: 2/0,0 mode: PR/PR res: [0x240000406:0xa:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x929304787231689 expref: -99 pid: 67243 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0
00010000:00010000:1.0:1775764890.050941:0:60134:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9f14902b3800 lock: ffff9f149817f000/0x23cab7dad48e5986 lrc: 2/0,0 mode: PR/PR res: [0x240000406:0x9:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x9293047872312b5 expref: -99 pid: 66165 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0
00010000:00010000:1.0:1775764890.051049:0:60134:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9f14902b3800 lock: ffff9f14a04c5200/0x23cab7dad48e5860 lrc: 2/0,0 mode: PR/PR res: [0x240000406:0x8:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x929304787230e08 expref: -99 pid: 65493 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0
00010000:00010000:1.0:1775764890.051179:0:60134:0:(ldlm_request.c:1438:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-mdc-ffff9f14902b3800 lock: ffff9f14c2152400/0x23cab7dad48e5baf lrc: 1/0,0 mode: --/PR res: [0x240000406:0xb:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x929304787231af7 expref: -99 pid: 68370 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0
00010000:00010000:1.0:1775764890.051189:0:60134:0:(ldlm_request.c:1438:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-mdc-ffff9f14902b3800 lock: ffff9f149817ee00/0x23cab7dad48e5a7b lrc: 1/0,0 mode: --/PR res: [0x240000406:0xa:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x929304787231689 expref: -99 pid: 67243 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0
00010000:00010000:1.0:1775764890.051196:0:60134:0:(ldlm_request.c:1438:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-mdc-ffff9f14902b3800 lock: ffff9f149817f000/0x23cab7dad48e5986 lrc: 1/0,0 mode: --/PR res: [0x240000406:0x9:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x9293047872312b5 expref: -99 pid: 66165 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0
00010000:00010000:1.0:1775764890.051202:0:60134:0:(ldlm_request.c:1438:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-mdc-ffff9f14902b3800 lock: ffff9f14a04c5200/0x23cab7dad48e5860 lrc: 1/0,0 mode: --/PR res: [0x240000406:0x8:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x929304787230e08 expref: -99 pid: 65493 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0
00010000:00010000:1.0:1775764890.051209:0:60134:0:(ldlm_request.c:1490:ldlm_cancel_pack()) 4 locks packed
00000100:00100000:1.0:1775764890.051236:0:60134:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fc700 pname:cluuid:pid:xid:nid:opc:job ldlm_bl_07:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:60134:1862022114192256:192.168.202.110@tcp:103:ldlm_bl.0
00000100:00100000:1.0:1775764890.052291:0:60134:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14887323c0 going to sleep for 11 seconds
00000100:00100000:1.0:1775764890.072442:0:60134:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fc700 pname:cluuid:pid:xid:nid:opc:job ldlm_bl_07:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:60134:1862022114192256:192.168.202.110@tcp:103:ldlm_bl.0
00010000:00010000:1.0:1775764890.072469:0:60134:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9f14902b3800 lock: ffff9f14c2152400/0x23cab7dad48e5baf lrc: 0/0,0 mode: --/PR res: [0x240000406:0xb:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x929304787231af7 expref: -99 pid: 68370 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0
00010000:00010000:1.0:1775764890.072491:0:60134:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9f14902b3800 lock: ffff9f149817ee00/0x23cab7dad48e5a7b lrc: 0/0,0 mode: --/PR res: [0x240000406:0xa:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x929304787231689 expref: -99 pid: 67243 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0
00010000:00010000:1.0:1775764890.072503:0:60134:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9f14902b3800 lock: ffff9f149817f000/0x23cab7dad48e5986 lrc: 0/0,0 mode: --/PR res: [0x240000406:0x9:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x9293047872312b5 expref: -99 pid: 66165 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0
00010000:00010000:1.0:1775764890.072514:0:60134:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9f14902b3800 lock: ffff9f14a04c5200/0x23cab7dad48e5860 lrc: 0/0,0 mode: --/PR res: [0x240000406:0x8:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x929304787230e08 expref: -99 pid: 65493 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0
00010000:00010000:1.0:1775764890.072540:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00010000:00010000:1.0:1775764890.072543:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00010000:00010000:1.0:1775764890.072545:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00010000:00010000:1.0:1775764890.136128:0:73837:0:(ldlm_resource.c:195:lru_size_store()) dropping all unused locks from namespace lustre-OST0000-osc-ffff9f14902b3800
00010000:00010000:1.0:1775764890.136169:0:73837:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 7/7 locks to regular blp list, 1 blwis in pool
00010000:00010000:1.0:1775764890.137020:0:73838:0:(ldlm_resource.c:195:lru_size_store()) dropping all unused locks from namespace lustre-OST0000-osc-ffff9f14b38c6800
00010000:00010000:1.0:1775764890.137064:0:73838:0:(ldlm_resource.c:195:lru_size_store()) dropping all unused locks from namespace lustre-OST0001-osc-ffff9f14902b3800
00010000:00010000:1.0:1775764890.137107:0:73838:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 7/7 locks to regular blp list, 1 blwis in pool
00000020:00010000:1.0:1775764891.955771:0:73979:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff9f1486124d58 (ffff9f1486143098/0) at cl_lock_release():245
00000020:00010000:1.0:1775764891.955776:0:73979:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9f1486124d58 (ffff9f1486143098/0) at cl_lock_cancel():142
00000020:00010000:1.0:1775764891.955779:0:73979:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9f14981b2430 (ffff9f1486143878/1) at cl_lock_cancel():142
00010000:00010000:1.0:1775764891.955789:0:73979:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-OST0000-osc-ffff9f14902b3800 lock: 00000000b9abcab7/0x23cab7dad48e631f lrc: 4/1,0 mode: PR/PR res: [0x280000401:0xbb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x810020000000000 nid: local remote: 0x92930478723362c expref: -99 pid: 73979 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0
00010000:00010000:1.0:1775764891.955803:0:73979:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-OST0000-osc-ffff9f14902b3800 lock: 00000000b9abcab7/0x23cab7dad48e631f lrc: 3/0,0 mode: PR/PR res: [0x280000401:0xbb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x800020000000000 nid: local remote: 0x92930478723362c expref: -99 pid: 73979 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0
00000020:00010000:1.0:1775764891.955820:0:73979:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9f1486124d58 (ffff9f1486143098/0) at cl_lock_fini():72
00000020:00010000:1.0:1775764891.955822:0:73979:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9f14981b2430 (ffff9f1486143098/1) at cl_lock_fini():72
00000080:00200000:1.0:1775764891.955831:0:73979:0:(vvp_io.c:517:vvp_io_advance()) advancing 4 bytes
00000080:00200000:1.0:1775764891.955853:0:73979:0:(vvp_io.c:288:vvp_io_fini()) [0x200000405:0x7e:0x0] ignore/verify layout 0/0, layout version 1 need write layout 0, restore needed 0, invalidate_lock 0
00000080:00200000:1.0:1775764891.955858:0:73979:0:(file.c:2206:ll_file_io_generic()) f209a.sanity-flr: 1 io complete with rc: 0, result: 4, restart: 0
00000080:00200000:1.0:1775764891.955863:0:73979:0:(file.c:2292:ll_file_io_generic()) iot: 1, result: 4
00000001:02000400:1.0:1775764894.706074:0:74115:0:(debug.c:645:libcfs_debug_mark_buffer()) DEBUG MARKER: first read complete
00000100:00100000:1.0:1775764894.814240:0:2396:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.110@tcp (source: 12345-192.168.202.110@tcp)
00000100:00100000:1.0:1775764894.815105:0:2403:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2403:1862022114195712:192.168.202.110@tcp:103:ldlm_bl.0
00010000:00010000:1.0:1775764894.988216:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00010000:00010000:1.0:1775764896.011576:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000080:00200008:1.0:1775764897.114709:0:73979:0:(file.c:5170:ll_file_seek()) START file f209a.sanity-flr:[0x200000405:0x7e:0x0], offset: 0, type: SEEK_SET
00000080:00200008:1.0:1775764897.114724:0:73979:0:(file.c:5201:ll_file_seek()) COMPLETED file f209a.sanity-flr:[0x200000405:0x7e:0x0], offset: 0, type: SEEK_SET, rc = 0
00000080:00200008:1.0:1775764897.114729:0:73979:0:(file.c:5170:ll_file_seek()) START file f209a.sanity-flr:[0x200000405:0x7e:0x0], offset: 0, type: SEEK_CUR
00000080:00200008:1.0:1775764897.114731:0:73979:0:(file.c:5201:ll_file_seek()) COMPLETED file f209a.sanity-flr:[0x200000405:0x7e:0x0], offset: 0, type: SEEK_CUR, rc = 0
00000080:00200008:1.0:1775764897.114737:0:73979:0:(file.c:2454:do_file_read_iter()) START file f209a.sanity-flr:[0x200000405:0x7e:0x0], ppos: 0, count: 4
00000080:00200000:1.0:1775764897.114775:0:73979:0:(file.c:2058:ll_file_io_generic()) f209a.sanity-flr: read ppos: 0, bytes: 4
00000080:00200000:1.0:1775764897.114779:0:73979:0:(file.c:1815:ll_io_set_mirror()) f209a.sanity-flr: desiginated mirror: 0
00000020:00200000:1.0:1775764897.114786:0:73979:0:(cl_io.c:214:cl_io_rw_init()) header@000000006410c142[0x110, 1, [0x200000405:0x7e:0x0] hash]
00000020:00200000:1.0:1775764897.114788:0:73979:0:(cl_io.c:214:cl_io_rw_init()) io range: 1 [0, 4) 0 0
00000080:00200000:1.0:1775764897.114793:0:73979:0:(vvp_io.c:1817:vvp_io_init()) [0x200000405:0x7e:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0
00000080:00010000:1.0:1775764897.114816:0:73979:0:(file.c:6988:ll_layout_intent()) ### lustre: requeue layout lock for file [0x200000405:0x7e:0x0](ffff9f14a10b00c8)
00000002:00010000:1.0:1775764897.114824:0:73979:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: ,[0x200000405:0x7e:0x0]) in obj [0x200000405:0x7e:0x0], intent: layout flags 0
00010000:00010000:1.0:1775764897.114872:0:73979:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f14a04c5600/0x23cab7dad48e6342 lrc: 3/1,0 mode: --/CR res: [0x200000405:0x7e:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 73979 initiator: MDT0 lru_score: 0 lru_type: 0
00010000:00010000:1.0:1775764897.114881:0:73979:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f14a04c5600/0x23cab7dad48e6342 lrc: 3/1,0 mode: --/CR res: [0x200000405:0x7e:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x0 pid: 73979 initiator: MDT0 lru_score: 0 lru_type: 0
00010000:00010000:1.0:1775764897.114890:0:73979:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f14a04c5600/0x23cab7dad48e6342 lrc: 3/1,0 mode: --/CR res: [0x200000405:0x7e:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x0 pid: 73979 initiator: MDT0 lru_score: 0 lru_type: 0
00000100:00100000:1.0:1775764897.114910:0:73979:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job multiop:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:73979:1862022114196224:192.168.202.110@tcp:101:multiop.0
00000100:00100000:1.0:1775764897.114989:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14813709c0 going to sleep for 50 seconds
00000100:00100000:1.0:1775764897.119130:0:73979:0:(client.c:3082:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9f14902b3800: committing for last_committed 8589936710 gen 1
00000100:00100000:1.0:1775764897.119166:0:73979:0:(client.c:3131:ptlrpc_free_committed()) @@@ keeping (FL_REPLAY)  req@ffff9f14ab4b5880 x1862022114193024/t8589936710(8589936710) o101->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 776/848 e 0 to 0 dl 1775764946 ref 1 fl Complete:RPQU/604/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0
00000100:00100000:1.0:1775764897.119178:0:73979:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job multiop:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:73979:1862022114196224:192.168.202.110@tcp:101:multiop.0
00010000:00010000:1.0:1775764897.119201:0:73979:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f14a04c5600/0x23cab7dad48e6342 lrc: 4/1,0 mode: CR/CR res: [0x200000405:0x7e:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x929304787233679 expref: -99 pid: 73979 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0
00010000:00010000:1.0:1775764897.119215:0:73979:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f14a04c5600/0x23cab7dad48e6342 lrc: 4/1,0 mode: CR/CR res: [0x200000405:0x7e:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x929304787233679 expref: -99 pid: 73979 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0
00000002:00100000:1.0:1775764897.119228:0:73979:0:(mdc_locks.c:833:mdc_finish_enqueue()) @@@ op=400 disposition=1, status=0  req@ffff9f14ab4ff100 x1862022114196224/t0(0) o101->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 376/584 e 0 to 0 dl 1775764952 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'multiop.0' uid:0 gid:0 projid:0
00000002:00010000:1.0:1775764897.119238:0:73979:0:(mdc_locks.c:937:mdc_finish_enqueue()) ### layout lock returned by: layout, lvb_len: 240 ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f14a04c5600/0x23cab7dad48e6342 lrc: 3/1,0 mode: CR/CR res: [0x200000405:0x7e:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x929304787233679 expref: -99 pid: 73979 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0
00000002:00010000:1.0:1775764897.119249:0:73979:0:(mdc_locks.c:1357:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f14a04c5600/0x23cab7dad48e6342 lrc: 3/1,0 mode: CR/CR res: [0x200000405:0x7e:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x929304787233679 expref: -99 pid: 73979 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0
00010000:00010000:1.0:1775764897.119262:0:73979:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9f1494571c00 type 13 mode 16 res 8589935621/126 (0 0)
00000080:00200000:1.0:1775764897.119268:0:73979:0:(namei.c:953:ll_intent_lock()) intent lock 1024 on i1 [0x200000405:0x7e:0x0] suppgids 0 0: rc 0
00000080:00010000:1.0:1775764897.119279:0:73979:0:(llite_internal.h:1984:ll_set_lock_data()) setting l_data to inode [0x200000405:0x7e:0x0](ffff9f14a10b00c8) for lock 0x23cab7dad48e6342
00000080:00010000:1.0:1775764897.119286:0:73979:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x23cab7dad48e6342 from it ffffb06c020d7af8
00010000:00010000:1.0:1775764897.119290:0:73979:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f14a04c5600/0x23cab7dad48e6342 lrc: 3/1,0 mode: CR/CR res: [0x200000405:0x7e:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x929304787233679 expref: -99 pid: 73979 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0
00010000:00010000:1.0:1775764897.119298:0:73979:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f14a04c5600/0x23cab7dad48e6342 lrc: 2/0,0 mode: CR/CR res: [0x200000405:0x7e:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x929304787233679 expref: -99 pid: 73979 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0
00010000:00010000:1.0:1775764897.119324:0:73979:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f14a04c5600/0x23cab7dad48e6342 lrc: 2/1,0 mode: CR/CR res: [0x200000405:0x7e:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x929304787233679 expref: -99 pid: 73979 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0
00010000:00010000:1.0:1775764897.119333:0:73979:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f14a04c5600/0x23cab7dad48e6342 lrc: 2/1,0 mode: CR/CR res: [0x200000405:0x7e:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x929304787233679 expref: -99 pid: 73979 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0
00000080:00010000:1.0:1775764897.119340:0:73979:0:(file.c:6871:ll_layout_lock_set()) ### file [0x200000405:0x7e:0x0](000000003993e00c) being reconfigured ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f14a04c5600/0x23cab7dad48e6342 lrc: 3/1,0 mode: CR/CR res: [0x200000405:0x7e:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x929304787233679 expref: -99 pid: 73979 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0
00010000:00010000:1.0:1775764897.119417:0:73979:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f14a04c5600/0x23cab7dad48e6342 lrc: 3/1,0 mode: CR/CR res: [0x200000405:0x7e:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x929304787233679 expref: -99 pid: 73979 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0
00010000:00010000:1.0:1775764897.119425:0:73979:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f14a04c5600/0x23cab7dad48e6342 lrc: 2/0,0 mode: CR/CR res: [0x200000405:0x7e:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x929304787233679 expref: -99 pid: 73979 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0
00000080:00200000:1.0:1775764897.119448:0:73979:0:(vvp_io.c:1817:vvp_io_init()) [0x200000405:0x7e:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0
00020000:00200000:1.0:1775764897.119457:0:73979:0:(lov_io.c:458:lov_io_mirror_init()) [0x200000405:0x7e:0x0]: flr state: 1, move mirror from 0 to 0, have retried: 0, mirror count: 2
00020000:00200000:1.0:1775764897.119459:0:73979:0:(lov_io.c:484:lov_io_mirror_init()) use delayed RPC state for this IO
00020000:00200000:1.0:1775764897.119465:0:73979:0:(lov_io.c:860:lov_io_iter_init()) component[0] flags 0x10
00000080:00200000:1.0:1775764897.119500:0:73979:0:(vvp_io.c:288:vvp_io_fini()) [0x200000405:0x7e:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0
00010000:00010000:1.0:1775764897.119517:0:73979:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f14a04c5600/0x23cab7dad48e6342 lrc: 2/1,0 mode: CR/CR res: [0x200000405:0x7e:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x929304787233679 expref: -99 pid: 73979 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0
00010000:00010000:1.0:1775764897.119525:0:73979:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f14a04c5600/0x23cab7dad48e6342 lrc: 2/1,0 mode: CR/CR res: [0x200000405:0x7e:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x929304787233679 expref: -99 pid: 73979 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0
00000080:00010000:1.0:1775764897.119530:0:73979:0:(file.c:6871:ll_layout_lock_set()) ### file [0x200000405:0x7e:0x0](000000003993e00c) being reconfigured ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f14a04c5600/0x23cab7dad48e6342 lrc: 3/1,0 mode: CR/CR res: [0x200000405:0x7e:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x929304787233679 expref: -99 pid: 73979 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0
00000080:00200000:1.0:1775764897.119900:0:73979:0:(vvp_io.c:1817:vvp_io_init()) [0x200000405:0x7e:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0
00020000:00200000:1.0:1775764897.119905:0:73979:0:(lov_io.c:458:lov_io_mirror_init()) [0x200000405:0x7e:0x0]: flr state: 1, move mirror from 0 to 0, have retried: 0, mirror count: 2
00020000:00200000:1.0:1775764897.119907:0:73979:0:(lov_io.c:484:lov_io_mirror_init()) use delayed RPC state for this IO
00020000:00200000:1.0:1775764897.119909:0:73979:0:(lov_io.c:860:lov_io_iter_init()) component[0] flags 0x10
00000080:00200000:1.0:1775764897.119933:0:73979:0:(vvp_io.c:288:vvp_io_fini()) [0x200000405:0x7e:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0
00000080:00200000:1.0:1775764897.119991:0:73979:0:(file.c:6766:ll_layout_conf()) [0x200000405:0x7e:0x0]: layout version change: 4294967294 -> 2
00010000:00010000:1.0:1775764897.120000:0:73979:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f14a04c5600/0x23cab7dad48e6342 lrc: 3/1,0 mode: CR/CR res: [0x200000405:0x7e:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x929304787233679 expref: -99 pid: 73979 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0
00010000:00010000:1.0:1775764897.120007:0:73979:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f14a04c5600/0x23cab7dad48e6342 lrc: 2/0,0 mode: CR/CR res: [0x200000405:0x7e:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x929304787233679 expref: -99 pid: 73979 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0
00020000:00200000:1.0:1775764897.120017:0:73979:0:(lov_io.c:458:lov_io_mirror_init()) [0x200000405:0x7e:0x0]: flr state: 2, move mirror from 0 to 0, have retried: 0, mirror count: 2
00020000:00200000:1.0:1775764897.120020:0:73979:0:(lov_io.c:484:lov_io_mirror_init()) use non-delayed RPC state for this IO
00020000:00200000:1.0:1775764897.120027:0:73979:0:(lov_io.c:1052:lov_io_rw_iter_init()) stripe: 0 chunk: [0, 4) 4, 4
00020000:00200000:1.0:1775764897.120029:0:73979:0:(lov_io.c:860:lov_io_iter_init()) component[0] flags 0x10
00000080:00200000:1.0:1775764897.120046:0:73979:0:(vvp_io.c:198:vvp_io_one_lock_index()) lock: 0 [0, 0]
00000008:00010000:1.0:1775764897.120056:0:73979:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 0
00000008:00010000:1.0:1775764897.120060:0:73979:0:(osc_lock.c:1310:osc_lock_init()) ### lock ffff9f14945e09b0, osc lock ffff9f148759a130, flags 0x400000000000000
00010000:00010000:1.0:1775764897.120097:0:73979:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-OST0000-osc-ffff9f14902b3800 lock: 00000000d6631618/0x23cab7dad48e6349 lrc: 3/1,0 mode: --/PR res: [0x280000401:0xbb:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 73979 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0
00010000:00010000:1.0:1775764897.120105:0:73979:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x400000000000000 ns: lustre-OST0000-osc-ffff9f14902b3800 lock: 00000000d6631618/0x23cab7dad48e6349 lrc: 3/1,0 mode: --/PR res: [0x280000401:0xbb:0x0].0x0 rrc: 2 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 73979 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0
00010000:00010000:1.0:1775764897.120116:0:73979:0:(ldlm_request.c:1139:ldlm_cli_enqueue()) @@@ enqueue lock with no delay  req@ffff9f14ab4ff100 x1862022114196352/t0(0) o101->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 328/400 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0
00010000:00010000:1.0:1775764897.120131:0:73979:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-OST0000-osc-ffff9f14902b3800 lock: 00000000d6631618/0x23cab7dad48e6349 lrc: 3/1,0 mode: --/PR res: [0x280000401:0xbb:0x0].0x0 rrc: 2 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 73979 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0
00000100:00100000:1.0:1775764897.120151:0:73979:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job multiop:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:73979:1862022114196352:192.168.202.110@tcp:101:multiop.0
00000100:00100000:1.0:1775764897.120224:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14813709c0 going to sleep for 11 seconds
00000100:00100000:1.0:1775764897.126956:0:2396:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.110@tcp (source: 12345-192.168.202.110@tcp)
00000008:00010000:1.0:1775764897.131548:0:2403:0:(osc_cache.c:901:osc_extent_finish()) ### extent: 00000000f2a0e3b7 ns: lustre-OST0000-osc-ffff9f14b38c6800 lock: 0000000041bbd801/0x23cab7dad48e633b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0xbb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x800429400020000 nid: local remote: 0x929304787233672 expref: -99 pid: 74118 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0
00000008:00010000:1.0:1775764897.131626:0:2403:0:(osc_cache.c:496:__osc_extent_remove()) ### extent: 00000000f2a0e3b7 ns: lustre-OST0000-osc-ffff9f14b38c6800 lock: 0000000041bbd801/0x23cab7dad48e633b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0xbb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x800429400020000 nid: local remote: 0x929304787233672 expref: -99 pid: 74118 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0
00000100:00100000:1.0:1775764897.131645:0:2403:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2403:1862022114196480:192.168.202.110@tcp:4:dd.0
00000100:00100000:1.0:1775764897.146007:0:44145:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1862022124564096
00000100:00100000:1.0:1775764897.146021:0:44145:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1862022124564096
00000100:00100000:1.0:1775764897.146076:0:44145:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.110@tcp, seq: 6456
00000100:00100000:1.0:1775764897.146084:0:44145:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9f14a04d1c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:LOV_OSC_UUID+4:10238:x1862022124564096:12345-192.168.202.110@tcp:105:
00010000:00010000:1.0:1775764897.146099:0:44145:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast  ns: lustre-OST0000-osc-ffff9f14902b3800 lock: 00000000d6631618/0x23cab7dad48e6349 lrc: 5/1,0 mode: --/PR res: [0x280000401:0xbb:0x0].0x0 rrc: 2 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x20000 nid: local remote: 0x929304787233680 expref: -99 pid: 73979 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0
00010000:00010000:1.0:1775764897.146112:0:44145:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-OST0000-osc-ffff9f14902b3800 lock: 00000000d6631618/0x23cab7dad48e6349 lrc: 5/1,0 mode: --/PR res: [0x280000401:0xbb:0x0].0x0 rrc: 2 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x20000 nid: local remote: 0x929304787233680 expref: -99 pid: 73979 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0
00010000:00010000:1.0:1775764897.146121:0:44145:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-OST0000-osc-ffff9f14902b3800 lock: 00000000d6631618/0x23cab7dad48e6349 lrc: 5/1,0 mode: --/PR res: [0x280000401:0xbb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x10000000020000 nid: local remote: 0x929304787233680 expref: -99 pid: 73979 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0
00010000:00010000:1.0:1775764897.146136:0:44145:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-OST0000-osc-ffff9f14902b3800 lock: 00000000d6631618/0x23cab7dad48e6349 lrc: 5/1,0 mode: PR/PR res: [0x280000401:0xbb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x10080000020000 nid: local remote: 0x929304787233680 expref: -99 pid: 73979 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0
00010000:00010000:1.0:1775764897.146144:0:44145:0:(ldlm_resource.c:1714:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-OST0000-osc-ffff9f14902b3800 lock: 00000000d6631618/0x23cab7dad48e6349 lrc: 6/1,0 mode: PR/PR res: [0x280000401:0xbb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x10080000020000 nid: local remote: 0x929304787233680 expref: -99 pid: 73979 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0
00010000:00010000:1.0:1775764897.146153:0:44145:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-OST0000-osc-ffff9f14902b3800 lock: 00000000d6631618/0x23cab7dad48e6349 lrc: 6/1,0 mode: PR/PR res: [0x280000401:0xbb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x80000020000 nid: local remote: 0x929304787233680 expref: -99 pid: 73979 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0
00010000:00010000:1.0:1775764897.146177:0:44145:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffff9f14a04c5e00)
00000100:00100000:1.0:1775764897.146241:0:44145:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9f14a04d1c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:LOV_OSC_UUID+4:10238:x1862022124564096:12345-192.168.202.110@tcp:105: Request processed in 157us (2364us total) trans 0 rc 0/0
00000100:00100000:1.0:1775764897.146248:0:44145:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.110@tcp, seq: 6456
00000100:00100000:1.0:1775764897.150529:0:2404:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2404:1862022114196608:192.168.202.110@tcp:103:ldlm_bl.0
00000100:00100000:1.0:1775764898.784869:0:2404:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2404:1862022114198528:192.168.202.110@tcp:3:multiop.0
00010000:00010000:1.0:1775764901.130748:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00010000:00010000:1.0:1775764902.154769:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775764903.434877:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1491687100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114199936:192.168.202.110@tcp:9:kworker.0
00010000:00010000:1.0:1775764908.298706:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00010000:00010000:1.0:1775764909.322750:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775764913.674685:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1775764903/real 1775764903]  req@ffff9f1491687100 x1862022114199936/t0(0) o9->lustre-OST0001-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 224/224 e 0 to 1 dl 1775764913 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775764913.674716:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1491687100 x1862022114199936/t0(0) o9->lustre-OST0001-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 224/224 e 0 to 1 dl 1775764913 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775764913.674736:0:2402:0:(import.c:2005:ptlrpc_disconnect_idle_interpret()) @@@ inflight=1, refcount=3: rc = -110  req@ffff9f1491687100 x1862022114199936/t0(0) o9->lustre-OST0001-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 224/224 e 0 to 1 dl 1775764913 ref 1 fl Interpret:EXNQU/200/ffffffff rc -110/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775764913.674746:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618800 lustre-OST0001_UUID: changing import state from CONNECTING to IDLE
00000100:00100000:1.0:1775764913.674750:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1491687100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114199936:192.168.202.110@tcp:9:kworker.0
00010000:00010000:1.0:1775764915.466726:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00010000:00010000:1.0:1775764916.490724:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775764918.794756:0:2403:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2403:1862022114202368:192.168.202.110@tcp:400:kworker.0
00000100:00100000:1.0:1775764918.794842:0:2403:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2403:1862022114202624:192.168.202.110@tcp:400:kworker.0
00000100:00100000:1.0:1775764918.794871:0:2403:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2403:1862022114202880:192.168.202.110@tcp:400:kworker.0
00000100:00100000:1.0:1775764918.794887:0:2403:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2403:1862022114203136:192.168.202.110@tcp:400:kworker.0
00000100:00080000:1.0:1775764919.834011:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775764919.834018:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1775764919.834021:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775764919.834027:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 1209
00000100:00080000:1.0:1775764919.834038:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775764919.834067:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdc00 x1862022114203392/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775764919.834106:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775764919.834108:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775764919.834110:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775764919.834113:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 1024
00000100:00080000:1.0:1775764919.834118:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775764919.834125:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775764919.834134:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ffb80 x1862022114203776/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775764919.834142:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775764919.834144:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1775764919.834145:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775764919.834148:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 1917
00000100:00080000:1.0:1775764919.834162:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775764919.834166:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775764919.834175:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fe300 x1862022114203904/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775764919.834193:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775764919.834197:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775764919.834199:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775764919.834205:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 2265
00000100:00080000:1.0:1775764919.834212:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775764919.834300:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775764919.834325:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff100 x1862022114204288/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775764919.834350:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level CONNECTING/4 force 1 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775764919.834355:0:37:0:(pinger.c:247:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: not pinging (in recovery or recovery disabled: CONNECTING)
00010000:00010000:1.0:1775764922.634719:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00010000:00010000:1.0:1775764923.658681:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1775764924.938679:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level CONNECTING/4 force 1 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775764924.938689:0:37:0:(pinger.c:247:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: not pinging (in recovery or recovery disabled: CONNECTING)
00000100:00080000:1.0:1775764924.938751:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775764924.938755:0:37:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING)
00000100:00080000:1.0:1775764924.938758:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775764924.938761:0:37:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: CONNECTING)
00000100:00080000:1.0:1775764924.938779:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775764924.938783:0:37:0:(pinger.c:247:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING)
00000100:00100000:1.0:1775764925.450641:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 1 seconds
00000100:00100000:1.0:1775764926.474651:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 1 seconds
00000100:00100000:1.0:1775764927.498645:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 1 seconds
00000100:00100000:1.0:1775764928.522672:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 1 seconds
00000100:00100000:1.0:1775764929.034680:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1775764919/real 1775764919]  req@ffff9f14ab4ff100 x1862022114204288/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775764929 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775764929.034735:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff100 x1862022114204288/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775764929 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775764929.034748:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1775764919/real 1775764919]  req@ffff9f14ab4ffb80 x1862022114203776/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775764929 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775764929.034772:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ffb80 x1862022114203776/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775764929 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775764929.034784:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1775764919/real 1775764919]  req@ffff9f14ab4fe300 x1862022114203904/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775764929 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775764929.034819:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fe300 x1862022114203904/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775764929 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775764929.034843:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1775764919/real 1775764919]  req@ffff9f14ab4fdc00 x1862022114203392/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775764929 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775764929.034856:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdc00 x1862022114203392/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775764929 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775764929.034898:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775764929.034911:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775764929.034926:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114204288:192.168.202.110@tcp:8:kworker.0
00000100:00080000:1.0:1775764929.034933:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775764929.034938:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775764929.034940:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ffb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114203776:192.168.202.110@tcp:8:kworker.0
00000100:00080000:1.0:1775764929.034945:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775764929.034951:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775764929.034955:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114203904:192.168.202.110@tcp:8:kworker.0
00000100:00080000:1.0:1775764929.034960:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775764929.034990:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775764929.034995:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114203392:192.168.202.110@tcp:250:kworker.0
00000100:00100000:1.0:1775764929.035225:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4b6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114204928:192.168.202.110@tcp:250:kworker.0
00000100:00100000:1.0:1775764929.035299:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4b7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114205312:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775764929.035339:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4b4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114205440:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775764929.035423:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4b5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114205824:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775764929.546656:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 1 seconds
00010000:00010000:1.0:1775764929.802695:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00000400:1.0:1775764930.570658:0:73979:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1775764914/real 1775764914]  req@ffff9f1485746d80 x1862022114202240/t0(0) o35->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:23/10 lens 392/2128 e 0 to 1 dl 1775764930 ref 2 fl Rpc:XQr/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0
00000100:00080000:1.0:1775764930.570690:0:73979:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from FULL to DISCONN
00000100:02000400:1.0:1775764930.570694:0:73979:0:(import.c:181:ptlrpc_set_import_discon()) lustre-MDT0000-mdc-ffff9f14902b3800: Connection to lustre-MDT0000 (at 192.168.202.110@tcp) was lost; in progress operations using this service will wait for recovery to complete
00000100:00080000:1.0:1775764930.575185:0:73979:0:(import.c:416:ptlrpc_pinger_force()) lustre-MDT0000_UUID: waking up pinger s:DISCONN
00000100:00100000:1.0:1775764930.575202:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:1.0:1775764930.575371:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4b6a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114205952:192.168.202.110@tcp:38:kworker.0
00010000:00010000:1.0:1775764930.826710:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775764931.594646:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:1.0:1775764932.618672:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:1.0:1775764933.642667:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:1.0:1775764934.666648:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:1.0:1775764935.690648:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:1.0:1775764936.714642:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775764936.970750:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775764937.738699:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775764937.994732:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775764938.762652:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:1.0:1775764939.786701:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:1.0:1775764940.746729:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1775764930/real 1775764930]  req@ffff9f14ab4b6a00 x1862022114205952/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775764940 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775764940.746777:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4b6a00 x1862022114205952/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775764940 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775764940.746806:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775764940.746844:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775764940.746850:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4b6a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114205952:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775764940.751019:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114207232:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775764940.810662:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:1.0:1775764941.834709:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:1.0:1775764942.858624:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:1.0:1775764943.882671:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775764944.138753:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775764944.842696:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1775764929/real 1775764929]  req@ffff9f14ab4b5500 x1862022114205824/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775764944 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775764944.842729:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4b5500 x1862022114205824/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775764944 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775764944.842739:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1775764929/real 1775764929]  req@ffff9f14ab4b4000 x1862022114205440/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775764944 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775764944.842750:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4b4000 x1862022114205440/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775764944 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775764944.842757:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1775764929/real 1775764929]  req@ffff9f14ab4b7480 x1862022114205312/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775764944 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775764944.842771:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4b7480 x1862022114205312/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775764944 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775764944.842780:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1775764929/real 1775764929]  req@ffff9f14ab4b6d80 x1862022114204928/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775764944 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775764944.842795:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4b6d80 x1862022114204928/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775764944 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775764944.842816:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775764944.842843:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775764944.842852:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4b5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114205824:192.168.202.110@tcp:8:kworker.0
00000100:00080000:1.0:1775764944.842879:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775764944.842908:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775764944.842915:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4b4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114205440:192.168.202.110@tcp:8:kworker.0
00000100:00080000:1.0:1775764944.842955:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775764944.842966:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775764944.842971:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4b7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114205312:192.168.202.110@tcp:8:kworker.0
00000100:00080000:1.0:1775764944.842979:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775764944.842996:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775764944.843003:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4b6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114204928:192.168.202.110@tcp:250:kworker.0
00000100:00100000:1.0:1775764944.843466:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114207744:192.168.202.110@tcp:250:kworker.0
00000100:00100000:1.0:1775764944.843512:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114208000:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775764944.843547:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114208128:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775764944.843598:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114208512:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775764944.906641:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775764945.162694:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775764945.930705:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:1.0:1775764946.954698:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:1.0:1775764947.978664:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:1.0:1775764949.002645:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:1.0:1775764950.026626:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:1.0:1775764951.050643:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775764951.306736:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775764952.074679:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775764952.330703:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775764953.098669:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:1.0:1775764954.122612:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:1.0:1775764955.082681:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1775764940/real 1775764940]  req@ffff9f14a04d0380 x1862022114207232/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775764955 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775764955.082725:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0380 x1862022114207232/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775764955 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775764955.082749:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775764955.082777:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775764955.082782:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114207232:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775764955.083878:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114209408:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775764955.146698:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:1.0:1775764956.170682:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:1.0:1775764957.194711:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:1.0:1775764958.042016:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775764958.042027:0:37:0:(pinger.c:247:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: not pinging (in recovery or recovery disabled: CONNECTING)
00000100:00080000:1.0:1775764958.042030:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775764958.042034:0:37:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING)
00000100:00080000:1.0:1775764958.042090:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775764958.042094:0:37:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING)
00000100:00080000:1.0:1775764958.042097:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775764958.042101:0:37:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: CONNECTING)
00000100:00080000:1.0:1775764958.042104:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775764958.042107:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775764958.042111:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775764958.042119:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 2265
00000100:00080000:1.0:1775764958.042136:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775764958.042149:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775764958.042175:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fe300 x1862022114210048/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775764958.042212:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775764958.042217:0:37:0:(pinger.c:247:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING)
00000100:00100000:1.0:1775764958.042251:0:2403:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2403:1862022114209920:192.168.202.110@tcp:400:kworker.0
00000100:00080000:1.0:1775764958.044542:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775764958.044583:0:37:0:(pinger.c:247:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: not pinging (in recovery or recovery disabled: CONNECTING)
00000100:00080000:1.0:1775764958.044588:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775764958.044593:0:37:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING)
00000100:00080000:1.0:1775764958.044597:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775764958.044601:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775764958.044604:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775764958.044610:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 1024
00000100:00080000:1.0:1775764958.044627:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775764958.044635:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775764958.044657:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff100 x1862022114210304/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775764958.044677:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775764958.044682:0:37:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING)
00000100:00080000:1.0:1775764958.044685:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775764958.044689:0:37:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: CONNECTING)
00000100:00080000:1.0:1775764958.044693:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775764958.044697:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775764958.044699:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775764958.044706:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 2265
00000100:00080000:1.0:1775764958.044716:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775764958.044723:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775764958.044742:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff480 x1862022114210432/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775764958.044756:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775764958.044760:0:37:0:(pinger.c:247:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING)
00000100:00100000:1.0:1775764958.218656:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775764958.474751:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775764959.242646:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775764959.498733:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775764960.266643:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:1.0:1775764961.290656:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:1.0:1775764962.314614:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:1.0:1775764963.082645:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775764963.082651:0:37:0:(pinger.c:247:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: not pinging (in recovery or recovery disabled: CONNECTING)
00000100:00080000:1.0:1775764963.082654:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775764963.082656:0:37:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING)
00000100:00080000:1.0:1775764963.082657:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775764963.082659:0:37:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: not pinging (in recovery or recovery disabled: CONNECTING)
00000100:00080000:1.0:1775764963.082660:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775764963.082662:0:37:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING)
00000100:00080000:1.0:1775764963.082663:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775764963.082665:0:37:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: CONNECTING)
00000100:00080000:1.0:1775764963.082666:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775764963.082670:0:37:0:(pinger.c:247:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING)
00000100:00080000:1.0:1775764963.082672:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775764963.082674:0:37:0:(pinger.c:247:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: not pinging (in recovery or recovery disabled: CONNECTING)
00000100:00080000:1.0:1775764963.082677:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775764963.082680:0:37:0:(pinger.c:247:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING)
00000100:00100000:1.0:1775764963.338638:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:1.0:1775764964.362595:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775764965.642775:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00010000:00010000:1.0:1775764966.666753:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1775764968.266911:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775764968.266919:0:37:0:(pinger.c:247:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: not pinging (in recovery or recovery disabled: CONNECTING)
00000100:00080000:1.0:1775764968.266923:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775764968.266927:0:37:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING)
00000100:00080000:1.0:1775764968.266930:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775764968.266934:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775764968.266938:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775764968.266946:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2340
00000100:00080000:1.0:1775764968.266960:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 5s
00000100:00080000:1.0:1775764968.266969:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775764968.266980:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775764968.267014:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fce00 x1862022114211072/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775764968.267048:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775764968.267053:0:37:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING)
00000100:00080000:1.0:1775764968.267056:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775764968.267060:0:37:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: CONNECTING)
00000100:00080000:1.0:1775764968.267063:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775764968.267067:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775764968.267069:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775764968.267074:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 2340
00000100:00080000:1.0:1775764968.267081:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 5s
00000100:00080000:1.0:1775764968.267088:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775764968.267093:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775764968.267105:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fe300 x1862022114211200/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775764968.267114:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775764968.267117:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775764968.267118:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775764968.267121:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 2340
00000100:00080000:1.0:1775764968.267139:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 5s
00000100:00080000:1.0:1775764968.267142:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775764968.267145:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775764968.267153:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff100 x1862022114211328/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775764968.267167:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775764968.267169:0:37:0:(pinger.c:247:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING)
00000100:00080000:1.0:1775764969.290947:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775764969.290967:0:37:0:(pinger.c:247:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: not pinging (in recovery or recovery disabled: CONNECTING)
00000100:00080000:1.0:1775764969.290970:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775764969.290980:0:37:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING)
00000100:00080000:1.0:1775764969.290984:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775764969.290987:0:37:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: not pinging (in recovery or recovery disabled: CONNECTING)
00000100:00080000:1.0:1775764969.290994:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775764969.290998:0:37:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING)
00000100:00080000:1.0:1775764969.291001:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775764969.291005:0:37:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: CONNECTING)
00000100:00080000:1.0:1775764969.291008:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775764969.291011:0:37:0:(pinger.c:247:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING)
00000100:00080000:1.0:1775764969.291014:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775764969.291018:0:37:0:(pinger.c:247:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: not pinging (in recovery or recovery disabled: CONNECTING)
00000100:00080000:1.0:1775764969.291021:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775764969.291024:0:37:0:(pinger.c:247:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING)
00010000:00010000:1.0:1775764972.810777:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1775764973.322606:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775764973.322613:0:37:0:(pinger.c:247:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: not pinging (in recovery or recovery disabled: CONNECTING)
00000100:00080000:1.0:1775764973.322615:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775764973.322617:0:37:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING)
00000100:00080000:1.0:1775764973.322618:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775764973.322620:0:37:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: not pinging (in recovery or recovery disabled: CONNECTING)
00000100:00080000:1.0:1775764973.322622:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775764973.322623:0:37:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING)
00000100:00080000:1.0:1775764973.322624:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775764973.322626:0:37:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: CONNECTING)
00000100:00080000:1.0:1775764973.322628:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775764973.322629:0:37:0:(pinger.c:247:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING)
00000100:00080000:1.0:1775764973.322631:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775764973.322632:0:37:0:(pinger.c:247:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: not pinging (in recovery or recovery disabled: CONNECTING)
00000100:00080000:1.0:1775764973.322634:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775764973.322636:0:37:0:(pinger.c:247:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING)
00000100:00000400:1.0:1775764973.578642:0:2404:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1775764918/real 1775764918]  req@ffff9f14a04d2300 x1862022114202496/t0(0) o400->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 224/224 e 0 to 1 dl 1775764973 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775764973.578682:0:2404:0:(client.c:2523:ptlrpc_expire_one_request()) @@@ TIMEOUT-NORESEND:  req@ffff9f14a04d2300 x1862022114202496/t0(0) o400->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 224/224 e 0 to 1 dl 1775764973 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775764973.578691:0:2404:0:(import.c:213:ptlrpc_set_import_discon()) mdc: import ffff9f14907d2000 already not connected (conn 13, was 10): CONNECTING
00000100:00000400:1.0:1775764973.578701:0:2404:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1775764918/real 1775764918]  req@ffff9f14a04d3800 x1862022114203008/t0(0) o400->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 224/224 e 0 to 1 dl 1775764973 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775764973.578719:0:2404:0:(client.c:2523:ptlrpc_expire_one_request()) @@@ TIMEOUT-NORESEND:  req@ffff9f14a04d3800 x1862022114203008/t0(0) o400->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 224/224 e 0 to 1 dl 1775764973 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775764973.578737:0:2404:0:(import.c:213:ptlrpc_set_import_discon()) mdc: import ffff9f14ab61b000 already not connected (conn 3, was 1): CONNECTING
00000100:00100000:1.0:1775764973.578755:0:2404:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2404:1862022114202496:192.168.202.110@tcp:400:kworker.0
00000100:00100000:1.0:1775764973.578763:0:2404:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2404:1862022114203008:192.168.202.110@tcp:400:kworker.0
00010000:00010000:1.0:1775764973.834686:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00000400:1.0:1775764979.722663:0:2404:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1775764924/real 1775764924]  req@ffff9f14ab4fdf80 x1862022114204416/t0(0) o400->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 224/224 e 0 to 1 dl 1775764979 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775764979.722703:0:2404:0:(client.c:2523:ptlrpc_expire_one_request()) @@@ TIMEOUT-NORESEND:  req@ffff9f14ab4fdf80 x1862022114204416/t0(0) o400->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 224/224 e 0 to 1 dl 1775764979 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775764979.722721:0:2404:0:(import.c:213:ptlrpc_set_import_discon()) mdc: import ffff9f14907d2000 already not connected (conn 15, was 10): DISCONN
00000100:00000400:1.0:1775764979.722728:0:2404:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1775764924/real 1775764924]  req@ffff9f14ab4fc700 x1862022114204672/t0(0) o400->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 224/224 e 0 to 1 dl 1775764979 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775764979.722740:0:2404:0:(client.c:2523:ptlrpc_expire_one_request()) @@@ TIMEOUT-NORESEND:  req@ffff9f14ab4fc700 x1862022114204672/t0(0) o400->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 224/224 e 0 to 1 dl 1775764979 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775764979.722745:0:2404:0:(import.c:213:ptlrpc_set_import_discon()) mdc: import ffff9f14ab61b000 already not connected (conn 4, was 1): DISCONN
00000100:00000400:1.0:1775764979.722749:0:2404:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1775764924/real 1775764924]  req@ffff9f14ab4fed80 x1862022114204800/t0(0) o400->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 224/224 e 0 to 1 dl 1775764979 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775764979.722761:0:2404:0:(client.c:2523:ptlrpc_expire_one_request()) @@@ TIMEOUT-NORESEND:  req@ffff9f14ab4fed80 x1862022114204800/t0(0) o400->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 224/224 e 0 to 1 dl 1775764979 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775764979.722765:0:2404:0:(import.c:213:ptlrpc_set_import_discon()) mdc: import ffff9f14ab619800 already not connected (conn 4, was 1): DISCONN
00000100:00000400:1.0:1775764979.722768:0:2404:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1775764924/real 1775764924]  req@ffff9f14ab4fd880 x1862022114204544/t0(0) o400->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 224/224 e 0 to 1 dl 1775764979 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775764979.722775:0:2404:0:(client.c:2523:ptlrpc_expire_one_request()) @@@ TIMEOUT-NORESEND:  req@ffff9f14ab4fd880 x1862022114204544/t0(0) o400->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 224/224 e 0 to 1 dl 1775764979 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775764979.722779:0:2404:0:(import.c:213:ptlrpc_set_import_discon()) mdc: import ffff9f14907d3000 already not connected (conn 4, was 1): DISCONN
00000100:00100000:1.0:1775764979.722789:0:2404:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2404:1862022114204416:192.168.202.110@tcp:400:kworker.0
00000100:00100000:1.0:1775764979.722792:0:2404:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fc700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2404:1862022114204672:192.168.202.110@tcp:400:kworker.0
00000100:00100000:1.0:1775764979.722795:0:2404:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2404:1862022114204800:192.168.202.110@tcp:400:kworker.0
00000100:00100000:1.0:1775764979.722798:0:2404:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2404:1862022114204544:192.168.202.110@tcp:400:kworker.0
00010000:00010000:1.0:1775764979.978742:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00010000:00010000:1.0:1775764981.002725:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1775764983.818675:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775764983.818686:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: not pinging (in recovery or recovery disabled: CONNECTING)
00000100:00080000:1.0:1775764983.818691:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775764983.818695:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775764983.818700:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775764983.818712:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2361
00000100:00080000:1.0:1775764983.818725:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775764983.818733:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775764983.818746:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775764983.818774:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fd880 x1862022114212096/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775764983.818794:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775764983.818797:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775764983.818798:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775764983.818800:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2361
00000100:00080000:1.0:1775764983.818804:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 15s
00000100:00080000:1.0:1775764983.818807:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775764983.818811:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775764983.818819:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fed80 x1862022114212224/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775764983.818835:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775764983.818838:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING)
00000100:00080000:1.0:1775764983.818839:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775764983.818841:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: CONNECTING)
00000100:00080000:1.0:1775764983.818843:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775764983.818845:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775764983.818846:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775764983.818848:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 2361
00000100:00080000:1.0:1775764983.818851:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 15s
00000100:00080000:1.0:1775764983.818854:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775764983.818858:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775764983.818865:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fc700 x1862022114212352/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775764983.818871:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775764983.818873:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775764983.818873:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775764983.818875:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 2361
00000100:00080000:1.0:1775764983.818878:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 15s
00000100:00080000:1.0:1775764983.818879:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775764983.818888:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775764983.818895:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdf80 x1862022114212480/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775764983.818902:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775764983.818905:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING)
00000100:00100000:1.0:1775764984.842660:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:1.0:1775764985.866655:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:1.0:1775764986.890728:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775764987.146721:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775764987.914663:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775764988.170662:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775764988.938665:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:1.0:1775764989.962672:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:1.0:1775764990.986656:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:1.0:1775764992.010615:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:1.0:1775764993.034637:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:1.0:1775764994.058646:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775764994.314735:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775764995.082627:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775764995.338698:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775764996.106749:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:1.0:1775764997.130683:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:1.0:1775764998.154761:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:1.0:1775764999.178675:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:1.0:1775765000.202763:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:1.0:1775765000.202804:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765000.202809:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1775765000.202813:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765000.202822:0:8:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 2377
00000100:00080000:1.0:1775765000.202833:0:8:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765000.202842:0:8:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775765000.202885:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff480 x1862022114215168/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765000.202921:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765000.202924:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775765000.202926:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765000.202931:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2377
00000100:00080000:1.0:1775765000.202938:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765000.202947:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765000.202960:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775765000.202982:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fd880 x1862022114215296/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765000.202995:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765000.203000:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775765000.203004:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765000.203009:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2377
00000100:00080000:1.0:1775765000.203015:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765000.203031:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765000.203039:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775765000.203056:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdf80 x1862022114215424/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765000.203068:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765000.203075:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775765000.203078:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765000.203085:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2377
00000100:00080000:1.0:1775765000.203092:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765000.203103:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765000.203115:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775765000.203134:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fc700 x1862022114215552/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765000.203166:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765000.203187:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1775765000.203191:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765000.203201:0:8:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2377
00000100:00080000:1.0:1775765000.203209:0:8:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765000.203218:0:8:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765000.203232:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775765000.203255:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fed80 x1862022114215680/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765000.203267:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765000.203278:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775765000.203283:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765000.203292:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 2377
00000100:00080000:1.0:1775765000.203297:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765000.203309:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765000.203317:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775765000.203342:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fce00 x1862022114215808/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765000.203373:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765000.203385:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775765000.203390:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765000.203399:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 2377
00000100:00080000:1.0:1775765000.203695:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765000.203711:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765000.203725:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775765000.203751:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff100 x1862022114215936/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765000.203793:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765000.203800:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775765000.203804:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765000.203816:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 2377
00000100:00080000:1.0:1775765000.203823:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765000.203834:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765000.203848:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775765000.203868:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fe300 x1862022114216064/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:1.0:1775765001.226729:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775765001.482662:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775765002.250628:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775765002.506718:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775765003.274638:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:1.0:1775765004.298660:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:1.0:1775765005.322680:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:1.0:1775765006.346676:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:1.0:1775765007.370648:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:1.0:1775765008.394676:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775765008.650712:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775765009.418694:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775765009.674724:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1775765010.442818:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765010.442826:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1775765010.442829:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765010.442837:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 2387
00000100:00080000:1.0:1775765010.442846:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765010.442852:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775765010.442894:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fca80 x1862022114217216/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765010.442914:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765010.442917:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775765010.442918:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765010.442921:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2387
00000100:00080000:1.0:1775765010.442925:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765010.442929:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765010.442939:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775765010.442956:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fea00 x1862022114217344/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765010.442972:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765010.442976:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775765010.442977:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765010.442983:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2387
00000100:00080000:1.0:1775765010.442987:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765010.442991:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765010.442997:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775765010.443009:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff800 x1862022114217472/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765010.443017:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765010.443020:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775765010.443021:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765010.443024:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2387
00000100:00080000:1.0:1775765010.443028:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765010.443030:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765010.443034:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775765010.443042:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fc380 x1862022114217600/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765010.443048:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765010.443051:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1775765010.443053:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765010.443057:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2387
00000100:00080000:1.0:1775765010.443059:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765010.443065:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765010.443073:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775765010.443098:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fe300 x1862022114217728/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765010.443110:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765010.443115:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775765010.443119:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765010.443128:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 2387
00000100:00080000:1.0:1775765010.443134:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765010.443138:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765010.443144:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775765010.443156:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff100 x1862022114217856/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765010.443175:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765010.443179:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775765010.443181:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765010.443187:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 2387
00000100:00080000:1.0:1775765010.443192:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765010.443198:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765010.443207:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775765010.443222:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fce00 x1862022114217984/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765010.443244:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765010.443247:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775765010.443249:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765010.443254:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 2387
00000100:00080000:1.0:1775765010.443259:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765010.443264:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765010.443274:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775765010.443286:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fed80 x1862022114218112/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00000400:1.0:1775765010.954721:0:2404:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1775764955/real 1775764955]  req@ffff9f14a04d3100 x1862022114209664/t0(0) o400->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 224/224 e 0 to 1 dl 1775765010 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765010.954762:0:2404:0:(client.c:2523:ptlrpc_expire_one_request()) @@@ TIMEOUT-NORESEND:  req@ffff9f14a04d3100 x1862022114209664/t0(0) o400->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 224/224 e 0 to 1 dl 1775765010 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765010.954770:0:2404:0:(import.c:213:ptlrpc_set_import_discon()) mdc: import ffff9f14ab61b000 already not connected (conn 11, was 1): CONNECTING
00000100:00000400:1.0:1775765010.954776:0:2404:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1775764955/real 1775764955]  req@ffff9f14a04d3b80 x1862022114209536/t0(0) o400->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 224/224 e 0 to 1 dl 1775765010 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765010.954786:0:2404:0:(client.c:2523:ptlrpc_expire_one_request()) @@@ TIMEOUT-NORESEND:  req@ffff9f14a04d3b80 x1862022114209536/t0(0) o400->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 224/224 e 0 to 1 dl 1775765010 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765010.954790:0:2404:0:(import.c:213:ptlrpc_set_import_discon()) mdc: import ffff9f14907d3000 already not connected (conn 11, was 1): CONNECTING
00000100:00000400:1.0:1775765010.954794:0:2404:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1775764955/real 1775764955]  req@ffff9f14a04d1f80 x1862022114209792/t0(0) o400->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 224/224 e 0 to 1 dl 1775765010 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765010.954801:0:2404:0:(client.c:2523:ptlrpc_expire_one_request()) @@@ TIMEOUT-NORESEND:  req@ffff9f14a04d1f80 x1862022114209792/t0(0) o400->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 224/224 e 0 to 1 dl 1775765010 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765010.954805:0:2404:0:(import.c:213:ptlrpc_set_import_discon()) mdc: import ffff9f14ab619800 already not connected (conn 11, was 1): CONNECTING
00000100:00100000:1.0:1775765010.954821:0:2404:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2404:1862022114209664:192.168.202.110@tcp:400:kworker.0
00000100:00100000:1.0:1775765010.954825:0:2404:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2404:1862022114209536:192.168.202.110@tcp:400:kworker.0
00000100:00100000:1.0:1775765010.954828:0:2404:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2404:1862022114209792:192.168.202.110@tcp:400:kworker.0
00000100:00000400:1.0:1775765013.002644:0:2404:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1775764958/real 1775764958]  req@ffff9f14ab4ffb80 x1862022114210176/t0(0) o400->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 224/224 e 0 to 1 dl 1775765013 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765013.002669:0:2404:0:(client.c:2523:ptlrpc_expire_one_request()) @@@ TIMEOUT-NORESEND:  req@ffff9f14ab4ffb80 x1862022114210176/t0(0) o400->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 224/224 e 0 to 1 dl 1775765013 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765013.002680:0:2404:0:(import.c:213:ptlrpc_set_import_discon()) mdc: import ffff9f14ab619800 already not connected (conn 11, was 1): CONNECTING
00000100:00100000:1.0:1775765013.002696:0:2404:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ffb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2404:1862022114210176:192.168.202.110@tcp:400:kworker.0
00000100:00000400:1.0:1775765013.514671:0:2403:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1775764958/real 1775764958]  req@ffff9f14ab4fdc00 x1862022114209920/t0(0) o400->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 224/224 e 0 to 1 dl 1775765013 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765013.514700:0:2403:0:(client.c:2523:ptlrpc_expire_one_request()) @@@ TIMEOUT-NORESEND:  req@ffff9f14ab4fdc00 x1862022114209920/t0(0) o400->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 224/224 e 0 to 1 dl 1775765013 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765013.514708:0:2403:0:(import.c:213:ptlrpc_set_import_discon()) mdc: import ffff9f14907d3000 already not connected (conn 11, was 1): CONNECTING
00000100:00100000:1.0:1775765013.514720:0:2403:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2403:1862022114209920:192.168.202.110@tcp:400:kworker.0
00010000:00010000:1.0:1775765015.818715:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00010000:00010000:1.0:1775765016.842711:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00010000:00010000:1.0:1775765022.986723:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00010000:00010000:1.0:1775765024.010731:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00010000:00010000:1.0:1775765030.154715:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00010000:00010000:1.0:1775765031.178730:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00010000:00010000:1.0:1775765037.322745:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00010000:00010000:1.0:1775765038.346748:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00010000:00010000:1.0:1775765044.490683:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00010000:00010000:1.0:1775765045.514672:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00010000:00010000:1.0:1775765051.658800:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00010000:00010000:1.0:1775765052.682665:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1775765056.522710:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765056.522721:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1775765056.522726:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765056.522740:0:8:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 2434
00000100:00080000:1.0:1775765056.522755:0:8:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765056.522779:0:8:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775765056.522831:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdc00 x1862022114226432/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765056.522866:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765056.522870:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775765056.522872:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765056.522878:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2434
00000100:00080000:1.0:1775765056.522898:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765056.522903:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765056.522917:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775765056.522937:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ffb80 x1862022114226560/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765056.522963:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765056.522968:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775765056.522971:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765056.522976:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2434
00000100:00080000:1.0:1775765056.522981:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765056.522985:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765056.522991:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775765056.523007:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fc700 x1862022114226688/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765056.523023:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765056.523031:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775765056.523034:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765056.523041:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2434
00000100:00080000:1.0:1775765056.523047:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765056.523053:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765056.523060:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775765056.523078:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdf80 x1862022114226816/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765056.523114:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765056.523131:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1775765056.523134:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765056.523140:0:8:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2434
00000100:00080000:1.0:1775765056.523145:0:8:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765056.523150:0:8:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765056.523157:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775765056.523187:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fd880 x1862022114226944/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765056.523210:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765056.523217:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775765056.523221:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765056.523238:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 2434
00000100:00080000:1.0:1775765056.523244:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765056.523255:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765056.523266:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775765056.523294:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff480 x1862022114227072/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765056.523310:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765056.523314:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775765056.523318:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765056.523323:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 2434
00000100:00080000:1.0:1775765056.523335:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765056.523345:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765056.523361:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775765056.523385:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdc00 x1862022114227200/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765056.523421:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765056.523434:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775765056.523438:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765056.523449:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 2434
00000100:00080000:1.0:1775765056.523454:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765056.523462:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765056.523478:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775765056.523503:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fca80 x1862022114227328/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00010000:00010000:1.0:1775765058.826764:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00010000:00010000:1.0:1775765059.850747:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775765060.618720:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:1.0:1775765061.642641:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:1.0:1775765062.666633:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:1.0:1775765063.690673:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:1.0:1775765064.714617:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:1.0:1775765065.738627:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775765065.994718:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775765066.762684:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775765067.018754:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775765067.786665:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:1.0:1775765068.810616:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:1.0:1775765069.834692:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:1.0:1775765070.858647:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:1.0:1775765071.882669:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765071.882680:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1775765071.882685:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765071.882696:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 2449
00000100:00080000:1.0:1775765071.882706:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765071.882720:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775765071.882769:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fc380 x1862022114229504/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765071.882804:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765071.882813:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775765071.882816:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765071.882821:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2449
00000100:00080000:1.0:1775765071.882825:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765071.882831:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765071.882842:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775765071.882860:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff800 x1862022114229632/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765071.882873:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765071.882877:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775765071.882879:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765071.882883:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2449
00000100:00080000:1.0:1775765071.882888:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765071.882892:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765071.882899:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775765071.882915:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fea00 x1862022114229760/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765071.882925:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765071.882929:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775765071.882931:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765071.882935:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2449
00000100:00080000:1.0:1775765071.882940:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765071.882950:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765071.882963:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775765071.882977:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff100 x1862022114229888/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765071.883002:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765071.883008:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1775765071.883011:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765071.883020:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2449
00000100:00080000:1.0:1775765071.883028:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765071.883034:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765071.883039:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775765071.883053:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fe300 x1862022114230016/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765071.883079:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765071.883083:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775765071.883085:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765071.883090:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 2449
00000100:00080000:1.0:1775765071.883096:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765071.883102:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765071.883110:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775765071.883125:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fce00 x1862022114230144/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765071.883145:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765071.883150:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775765071.883152:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765071.883161:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 2449
00000100:00080000:1.0:1775765071.883168:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765071.883175:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765071.883181:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775765071.883197:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fed80 x1862022114230272/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765071.883228:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765071.883231:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775765071.883233:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765071.883238:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 2449
00000100:00080000:1.0:1775765071.883244:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765071.883248:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765071.883254:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775765071.883272:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fc380 x1862022114230400/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00010000:00010000:1.0:1775765073.162837:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00010000:00010000:1.0:1775765074.186750:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775765075.978739:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:1.0:1775765077.002661:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:1.0:1775765078.026642:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:1.0:1775765079.050743:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:1.0:1775765080.074650:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775765080.330758:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775765081.098723:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775765081.354751:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775765082.122674:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:1.0:1775765082.122800:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114231552:192.168.202.110@tcp:250:kworker.0
00000800:00000100:1.0:1775765082.122860:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765082.122897:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765082/real 1775765082]  req@ffff9f14a04d2300 x1862022114231552/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765107 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765082.122916:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d2300 x1862022114231552/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765107 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765082.122930:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765082.122935:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765082.122942:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114231552:192.168.202.110@tcp:250:kworker.0
00000100:00100000:1.0:1775765082.122973:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114231680:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765082.123000:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765082.123012:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765082/real 1775765082]  req@ffff9f14a04d0e00 x1862022114231680/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765107 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765082.123044:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0e00 x1862022114231680/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765107 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765082.123065:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765082.123073:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765082.123076:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114231680:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765082.123087:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114231808:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765082.123111:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765082.123123:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765082/real 1775765082]  req@ffff9f14a04d2a00 x1862022114231808/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765107 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765082.123142:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d2a00 x1862022114231808/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765107 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765082.123160:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765082.123168:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765082.123174:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114231808:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765082.123183:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114231936:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765082.123204:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765082.123216:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765082/real 1775765082]  req@ffff9f14a04d0a80 x1862022114231936/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765107 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765082.123229:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0a80 x1862022114231936/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765107 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765082.123236:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765082.123240:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765082.123242:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114231936:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775765082.123246:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114232064:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765082.123257:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765082.123264:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765082/real 1775765082]  req@ffff9f14a04d0380 x1862022114232064/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765107 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765082.123272:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0380 x1862022114232064/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765107 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765082.123278:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765082.123280:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765082.123282:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114232064:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775765082.123295:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114232192:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765082.123304:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765082.123309:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765082/real 1775765082]  req@ffff9f14a04d1f80 x1862022114232192/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765107 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765082.123317:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1f80 x1862022114232192/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765107 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765082.123322:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765082.123323:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765082.123325:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114232192:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765082.123329:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114232320:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765082.123337:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765082.123342:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765082/real 1775765082]  req@ffff9f14a04d3b80 x1862022114232320/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765107 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765082.123349:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3b80 x1862022114232320/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765107 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765082.123354:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765082.123356:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765082.123357:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114232320:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765082.123360:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114232448:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765082.123368:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765082.123373:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765082/real 1775765082]  req@ffff9f14a04d3100 x1862022114232448/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765107 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765082.123379:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3100 x1862022114232448/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765107 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765082.123389:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765082.123391:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765082.123393:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114232448:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775765083.146657:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:1.0:1775765084.170649:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:1.0:1775765085.194757:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:1.0:1775765086.218670:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:1.0:1775765087.242680:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765087.242690:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1775765087.242694:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765087.242703:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 2464
00000100:00080000:1.0:1775765087.242715:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765087.242723:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775765087.242760:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fca80 x1862022114232576/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765087.242783:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765087.242786:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775765087.242788:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765087.242791:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2464
00000100:00080000:1.0:1775765087.242794:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765087.242797:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765087.242805:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775765087.242815:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdc00 x1862022114232704/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765087.242826:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765087.242835:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775765087.242837:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765087.242841:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2464
00000100:00080000:1.0:1775765087.242843:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765087.242845:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765087.242848:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775765087.242861:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff480 x1862022114232832/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765087.242871:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765087.242875:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775765087.242877:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765087.242882:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2464
00000100:00080000:1.0:1775765087.242887:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765087.242892:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765087.242901:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775765087.242918:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fd880 x1862022114232960/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765087.242926:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765087.242929:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1775765087.242930:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765087.242934:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2464
00000100:00080000:1.0:1775765087.242940:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765087.242947:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765087.242954:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775765087.242967:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdf80 x1862022114233088/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765087.242976:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765087.242985:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775765087.242988:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765087.242996:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 2464
00000100:00080000:1.0:1775765087.243001:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765087.243004:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765087.243007:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775765087.243018:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fc700 x1862022114233216/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765087.243028:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765087.243035:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775765087.243037:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765087.243041:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 2464
00000100:00080000:1.0:1775765087.243043:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765087.243047:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765087.243052:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775765087.243065:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ffb80 x1862022114233344/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765087.243085:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765087.243090:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775765087.243093:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765087.243100:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 2464
00000100:00080000:1.0:1775765087.243109:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765087.243114:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765087.243121:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775765087.243139:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fca80 x1862022114233472/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00010000:00010000:1.0:1775765087.498723:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00010000:00010000:1.0:1775765088.522735:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00010000:00010000:1.0:1775765094.666694:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00010000:00010000:1.0:1775765095.690682:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775765097.483005:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114234624:192.168.202.110@tcp:250:kworker.0
00000800:00000100:1.0:1775765097.483139:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765097.483209:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765097/real 1775765097]  req@ffff9f1485744a80 x1862022114234624/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765122 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765097.483238:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744a80 x1862022114234624/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765122 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765097.483252:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765097.483257:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765097.483263:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114234624:192.168.202.110@tcp:250:kworker.0
00000100:00100000:1.0:1775765097.483271:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114234752:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765097.483295:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765097.483307:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765097/real 1775765097]  req@ffff9f1485747b80 x1862022114234752/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765122 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765097.483334:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747b80 x1862022114234752/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765122 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765097.483341:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765097.483344:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765097.483347:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114234752:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765097.483373:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114234880:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765097.483387:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765097.483394:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765097/real 1775765097]  req@ffff9f1485744e00 x1862022114234880/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765122 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765097.483404:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744e00 x1862022114234880/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765122 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765097.483410:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765097.483429:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765097.483432:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114234880:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765097.483437:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114235008:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765097.483452:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765097.483459:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765097/real 1775765097]  req@ffff9f1485745880 x1862022114235008/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765122 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765097.483475:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745880 x1862022114235008/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765122 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765097.483481:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765097.483484:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765097.483486:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114235008:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775765097.483490:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114235136:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765097.483501:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765097.483598:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765097/real 1775765097]  req@ffff9f1485746680 x1862022114235136/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765122 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765097.483608:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746680 x1862022114235136/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765122 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765097.483631:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765097.483633:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765097.483636:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114235136:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775765097.483641:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114235264:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765097.483662:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765097.483671:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765097/real 1775765097]  req@ffff9f1485746a00 x1862022114235264/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765122 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765097.483680:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746a00 x1862022114235264/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765122 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765097.483692:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765097.483695:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765097.483698:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114235264:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765097.483702:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114235392:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765097.483714:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765097.483721:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765097/real 1775765097]  req@ffff9f1485744380 x1862022114235392/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765122 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765097.483730:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744380 x1862022114235392/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765122 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765097.483737:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765097.483739:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765097.483741:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114235392:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765097.483745:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114235520:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765097.483757:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765097.483763:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765097/real 1775765097]  req@ffff9f1485744700 x1862022114235520/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765122 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765097.483771:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744700 x1862022114235520/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765122 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765097.483795:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765097.483797:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765097.483800:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114235520:192.168.202.110@tcp:8:kworker.0
00010000:00010000:1.0:1775765101.834678:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775765102.602881:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114235648:192.168.202.110@tcp:250:kworker.0
00000800:00000100:1.0:1775765102.602963:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765102.603001:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765102/real 1775765102]  req@ffff9f1485745f80 x1862022114235648/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765127 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765102.603019:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745f80 x1862022114235648/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765127 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765102.603044:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765102.603052:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765102.603061:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114235648:192.168.202.110@tcp:250:kworker.0
00000100:00100000:1.0:1775765102.603072:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114235776:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765102.603096:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765102.603109:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765102/real 1775765102]  req@ffff9f1485746300 x1862022114235776/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765127 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765102.603131:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746300 x1862022114235776/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765127 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765102.603142:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765102.603147:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765102.603153:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114235776:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765102.603164:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114235904:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765102.603180:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765102.603187:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765102/real 1775765102]  req@ffff9f1485745c00 x1862022114235904/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765127 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765102.603196:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745c00 x1862022114235904/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765127 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765102.603201:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765102.603203:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765102.603205:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114235904:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765102.603223:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114236032:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765102.603233:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765102.603248:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765102/real 1775765102]  req@ffff9f1485745500 x1862022114236032/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765127 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765102.603255:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745500 x1862022114236032/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765127 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765102.603261:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765102.603263:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765102.603264:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114236032:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775765102.603268:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114236160:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765102.603278:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765102.603283:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765102/real 1775765102]  req@ffff9f1485747480 x1862022114236160/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765127 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765102.603295:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747480 x1862022114236160/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765127 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765102.603300:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765102.603301:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765102.603303:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114236160:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775765102.603307:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114236288:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765102.603319:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765102.603327:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765102/real 1775765102]  req@ffff9f1485747100 x1862022114236288/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765127 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765102.603334:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747100 x1862022114236288/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765127 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765102.603339:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765102.603340:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765102.603342:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114236288:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765102.603345:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114236416:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765102.603354:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765102.603359:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765102/real 1775765102]  req@ffff9f1485744000 x1862022114236416/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765127 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765102.603365:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744000 x1862022114236416/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765127 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765102.603374:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765102.603378:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765102.603380:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114236416:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765102.603384:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114236544:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765102.603394:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765102.603399:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765102/real 1775765102]  req@ffff9f1485747800 x1862022114236544/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765127 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765102.603406:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747800 x1862022114236544/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765127 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765102.603410:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765102.603412:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765102.603414:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114236544:192.168.202.110@tcp:8:kworker.0
00010000:00010000:1.0:1775765102.858764:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00010000:00010000:1.0:1775765109.002696:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00010000:00010000:1.0:1775765110.026691:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00010000:00010000:1.0:1775765116.170787:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00010000:00010000:1.0:1775765117.194747:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1775765117.962728:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765117.962739:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1775765117.962744:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765117.962755:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 2495
00000100:00080000:1.0:1775765117.962771:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765117.962781:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775765117.962831:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fc380 x1862022114238720/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765117.962858:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765117.962862:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775765117.962866:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765117.962872:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2495
00000100:00080000:1.0:1775765117.962877:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765117.962884:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765117.962896:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775765117.962921:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fed80 x1862022114238848/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765117.962934:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765117.962939:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775765117.962942:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765117.962947:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2495
00000100:00080000:1.0:1775765117.962966:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765117.962983:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765117.962991:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775765117.963012:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fce00 x1862022114238976/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765117.963024:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765117.963028:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775765117.963030:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765117.963034:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2495
00000100:00080000:1.0:1775765117.963038:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765117.963043:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765117.963060:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775765117.963074:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fe300 x1862022114239104/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765117.963098:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765117.963104:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1775765117.963105:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765117.963109:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2495
00000100:00080000:1.0:1775765117.963116:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765117.963120:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765117.963124:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775765117.963134:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff100 x1862022114239232/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765117.963150:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765117.963153:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775765117.963159:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765117.963174:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 2495
00000100:00080000:1.0:1775765117.963183:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765117.963195:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765117.963204:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775765117.963226:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fea00 x1862022114239360/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765117.963246:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765117.963258:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775765117.963259:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765117.963264:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 2495
00000100:00080000:1.0:1775765117.963268:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765117.963275:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765117.963288:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775765117.963298:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff800 x1862022114239488/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765117.963305:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765117.963308:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775765117.963309:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765117.963311:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 2495
00000100:00080000:1.0:1775765117.963330:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765117.963337:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765117.963349:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775765117.963365:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fc380 x1862022114239616/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00010000:00010000:1.0:1775765123.338873:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00010000:00010000:1.0:1775765130.506736:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000400:00000100:1.0:1775765136.395013:0:2390:0:(lib-socket.c:454:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.202.110:988
00000400:00000100:1.0:1775765136.395076:0:2390:0:(acceptor.c:111:lnet_connect_console_error()) Connection to 192.168.202.110@tcp at host 192.168.202.110 was unreachable: the network or that node may be down, or Lustre may be misconfigured.
00000800:00000100:1.0:1775765136.395103:0:2390:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:1.0:1775765136.395195:0:2390:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:1.0:1775765136.395212:0:2390:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:1.0:1775765136.395225:0:2390:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:1.0:1775765136.395245:0:2390:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:1.0:1775765136.395273:0:2390:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:1.0:1775765136.395293:0:2390:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:1.0:1775765136.395324:0:2390:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00010000:00010000:1.0:1775765137.674820:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00010000:00010000:1.0:1775765144.842707:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00010000:00010000:1.0:1775765152.010827:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00010000:00010000:1.0:1775765159.178791:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00010000:00010000:1.0:1775765166.346698:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00010000:00010000:1.0:1775765173.514828:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00010000:00010000:1.0:1775765175.562714:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00010000:00010000:1.0:1775765180.682725:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00010000:00010000:1.0:1775765182.730690:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1775765184.522754:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765184.522762:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1775765184.522767:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765184.522777:0:8:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 2562
00000100:00080000:1.0:1775765184.522789:0:8:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765184.522797:0:8:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775765184.522842:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fca80 x1862022114252032/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765184.522875:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765184.522878:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775765184.522880:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765184.522884:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2562
00000100:00080000:1.0:1775765184.522887:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765184.522889:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765184.522899:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775765184.522915:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ffb80 x1862022114252160/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765184.522924:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765184.522928:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775765184.522932:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765184.522947:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2562
00000100:00080000:1.0:1775765184.522949:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765184.522952:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765184.522956:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775765184.522973:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fc700 x1862022114252288/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765184.522980:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765184.522982:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775765184.522984:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765184.522987:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2562
00000100:00080000:1.0:1775765184.523001:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765184.523004:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765184.523007:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775765184.523019:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdf80 x1862022114252416/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765184.523035:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765184.523039:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1775765184.523040:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765184.523046:0:8:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2562
00000100:00080000:1.0:1775765184.523052:0:8:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765184.523056:0:8:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765184.523061:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775765184.523079:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fd880 x1862022114252544/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765184.523086:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765184.523096:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775765184.523099:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765184.523117:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 2562
00000100:00080000:1.0:1775765184.523124:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765184.523129:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765184.523135:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775765184.523156:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff480 x1862022114252672/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765184.523175:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765184.523180:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775765184.523183:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765184.523193:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 2562
00000100:00080000:1.0:1775765184.523205:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765184.523214:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765184.523231:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775765184.523249:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdc00 x1862022114252800/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765184.523291:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765184.523301:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775765184.523303:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765184.523309:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 2562
00000100:00080000:1.0:1775765184.523313:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765184.523318:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765184.523326:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775765184.523338:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fca80 x1862022114252928/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00010000:00010000:1.0:1775765187.850736:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775765189.642832:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114253056:192.168.202.110@tcp:250:kworker.0
00000800:00000100:1.0:1775765189.642895:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765189.642927:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765189/real 1775765189]  req@ffff9f1485746300 x1862022114253056/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765214 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765189.642940:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746300 x1862022114253056/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765214 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765189.642948:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765189.642950:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765189.642954:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114253056:192.168.202.110@tcp:250:kworker.0
00000100:00100000:1.0:1775765189.642971:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114253184:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765189.642988:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765189.642999:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765189/real 1775765189]  req@ffff9f1485744700 x1862022114253184/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765214 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765189.643010:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744700 x1862022114253184/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765214 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765189.643015:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765189.643017:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765189.643019:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114253184:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765189.643183:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114253312:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765189.643213:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765189.643234:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765189/real 1775765189]  req@ffff9f1485744380 x1862022114253312/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765214 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765189.643264:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744380 x1862022114253312/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765214 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765189.643283:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765189.643289:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765189.643295:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114253312:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765189.643313:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114253440:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765189.643335:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765189.643349:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765189/real 1775765189]  req@ffff9f1485746a00 x1862022114253440/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765214 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765189.643368:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746a00 x1862022114253440/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765214 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765189.643378:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765189.643383:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765189.643387:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114253440:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775765189.643399:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114253568:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765189.643416:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765189.643428:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765189/real 1775765189]  req@ffff9f1485746680 x1862022114253568/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765214 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765189.643446:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746680 x1862022114253568/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765214 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765189.643457:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765189.643464:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765189.643467:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114253568:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775765189.643478:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114253696:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765189.643499:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765189.643510:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765189/real 1775765189]  req@ffff9f1485745880 x1862022114253696/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765214 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765189.643524:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745880 x1862022114253696/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765214 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765189.643536:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765189.643543:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765189.643546:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114253696:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765189.643578:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114253824:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765189.643596:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765189.643609:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765189/real 1775765189]  req@ffff9f1485744e00 x1862022114253824/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765214 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765189.643618:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744e00 x1862022114253824/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765214 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765189.643624:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765189.643627:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765189.643630:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114253824:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765189.643637:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114253952:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765189.643648:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765189.643671:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765189/real 1775765189]  req@ffff9f1485747b80 x1862022114253952/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765214 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765189.643679:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747b80 x1862022114253952/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765214 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765189.643685:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765189.643687:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765189.643689:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114253952:192.168.202.110@tcp:8:kworker.0
00010000:00010000:1.0:1775765189.898700:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775765194.762835:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114254080:192.168.202.110@tcp:250:kworker.0
00000800:00000100:1.0:1775765194.762899:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765194.762948:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765194/real 1775765194]  req@ffff9f1485744a80 x1862022114254080/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765219 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765194.762985:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744a80 x1862022114254080/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765219 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765194.763010:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765194.763020:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765194.763035:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114254080:192.168.202.110@tcp:250:kworker.0
00000100:00100000:1.0:1775765194.763046:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114254208:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765194.763079:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765194.763099:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765194/real 1775765194]  req@ffff9f1485747b80 x1862022114254208/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765219 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765194.763125:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747b80 x1862022114254208/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765219 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765194.763131:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765194.763134:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765194.763136:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114254208:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765194.763154:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114254336:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765194.763172:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765194.763180:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765194/real 1775765194]  req@ffff9f1485744e00 x1862022114254336/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765219 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765194.763188:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744e00 x1862022114254336/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765219 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765194.763194:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765194.763196:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765194.763198:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114254336:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765194.763202:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114254464:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765194.763212:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765194.763217:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765194/real 1775765194]  req@ffff9f1485745880 x1862022114254464/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765219 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765194.763240:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745880 x1862022114254464/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765219 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765194.763249:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765194.763253:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765194.763256:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114254464:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775765194.763263:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114254592:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765194.763283:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765194.763294:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765194/real 1775765194]  req@ffff9f1485746680 x1862022114254592/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765219 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765194.763308:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746680 x1862022114254592/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765219 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765194.763313:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765194.763315:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765194.763318:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114254592:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775765194.763321:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114254720:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765194.763333:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765194.763340:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765194/real 1775765194]  req@ffff9f1485746a00 x1862022114254720/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765219 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765194.763347:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746a00 x1862022114254720/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765219 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765194.763352:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765194.763354:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765194.763356:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114254720:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765194.763359:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114254848:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765194.763369:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765194.763374:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765194/real 1775765194]  req@ffff9f1485744380 x1862022114254848/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765219 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765194.763381:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744380 x1862022114254848/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765219 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765194.763399:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765194.763401:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765194.763403:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114254848:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765194.763407:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114254976:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765194.763418:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765194.763424:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765194/real 1775765194]  req@ffff9f1485744700 x1862022114254976/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765219 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765194.763432:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744700 x1862022114254976/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765219 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765194.763437:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765194.763438:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765194.763440:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114254976:192.168.202.110@tcp:8:kworker.0
00010000:00010000:1.0:1775765195.018762:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00010000:00010000:1.0:1775765197.066725:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1775765199.882680:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765199.882688:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1775765199.882693:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765199.882703:0:8:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 2577
00000100:00080000:1.0:1775765199.882714:0:8:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765199.882721:0:8:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775765199.882758:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fc380 x1862022114255104/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765199.882796:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765199.882800:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775765199.882802:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765199.882807:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2577
00000100:00080000:1.0:1775765199.882811:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765199.882816:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765199.882825:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775765199.882840:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff800 x1862022114255232/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765199.882858:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765199.882862:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775765199.882864:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765199.882870:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2577
00000100:00080000:1.0:1775765199.882874:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765199.882878:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765199.882884:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775765199.882896:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fea00 x1862022114255360/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765199.882918:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765199.882922:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775765199.882924:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765199.882929:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2577
00000100:00080000:1.0:1775765199.882935:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765199.882940:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765199.882947:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775765199.882965:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff100 x1862022114255488/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765199.882977:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765199.882987:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1775765199.882989:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765199.882995:0:8:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2577
00000100:00080000:1.0:1775765199.883000:0:8:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765199.883006:0:8:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765199.883015:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775765199.883030:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fe300 x1862022114255616/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765199.883055:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765199.883059:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775765199.883061:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765199.883067:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 2577
00000100:00080000:1.0:1775765199.883072:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765199.883078:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765199.883085:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775765199.883101:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fce00 x1862022114255744/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765199.883124:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765199.883128:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775765199.883131:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765199.883136:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 2577
00000100:00080000:1.0:1775765199.883143:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765199.883149:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765199.883155:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775765199.883168:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fed80 x1862022114255872/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765199.883190:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765199.883194:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775765199.883197:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765199.883201:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 2577
00000100:00080000:1.0:1775765199.883206:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765199.883215:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765199.883221:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775765199.883235:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fca80 x1862022114256000/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00010000:00010000:1.0:1775765202.186704:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775765202.955220:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765199/real 1775765202]  req@ffff9f14ab4fce00 x1862022114255744/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765224 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765202.955251:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fce00 x1862022114255744/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765224 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765202.955264:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765202.955271:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765202.955275:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114255744:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765202.955284:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765199/real 1775765202]  req@ffff9f14ab4fe300 x1862022114255616/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765224 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765202.955297:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fe300 x1862022114255616/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765224 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765202.955305:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765202.955310:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765202.955314:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114255616:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775765202.955321:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765199/real 1775765202]  req@ffff9f14ab4ff100 x1862022114255488/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765224 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765202.955334:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff100 x1862022114255488/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765224 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765202.955342:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765202.955345:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765202.955349:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114255488:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775765202.955356:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765199/real 1775765202]  req@ffff9f14ab4ff800 x1862022114255232/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765224 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765202.955369:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff800 x1862022114255232/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765224 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765202.955379:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765202.955383:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765202.955387:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114255232:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765202.955395:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765199/real 1775765202]  req@ffff9f14ab4fea00 x1862022114255360/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765224 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765202.955414:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fea00 x1862022114255360/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765224 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765202.955422:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765202.955428:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765202.955431:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114255360:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765202.955439:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765199/real 1775765202]  req@ffff9f14ab4fc380 x1862022114255104/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765224 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765202.955451:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fc380 x1862022114255104/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765224 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765202.955459:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765202.955463:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765202.955466:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114255104:192.168.202.110@tcp:250:kworker.0
00000100:00100000:1.0:1775765202.955506:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765199/real 1775765202]  req@ffff9f14ab4fca80 x1862022114256000/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765224 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765202.955520:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fca80 x1862022114256000/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765224 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765202.955545:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765202.955548:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765202.955589:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114256000:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775765202.955597:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765199/real 1775765202]  req@ffff9f14ab4fed80 x1862022114255872/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765224 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765202.955612:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fed80 x1862022114255872/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765224 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765202.955626:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765202.955630:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765202.955633:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114255872:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765205.002849:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114256128:192.168.202.110@tcp:250:kworker.0
00000800:00000100:1.0:1775765205.002944:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765205.003007:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765205/real 1775765205]  req@ffff9f1485746300 x1862022114256128/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765230 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765205.003041:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746300 x1862022114256128/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765230 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765205.003068:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765205.003078:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765205.003088:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114256128:192.168.202.110@tcp:250:kworker.0
00000100:00100000:1.0:1775765205.003141:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114256256:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765205.003189:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765205.003206:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765205/real 1775765205]  req@ffff9f1485745c00 x1862022114256256/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765230 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765205.003236:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745c00 x1862022114256256/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765230 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765205.003252:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765205.003261:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765205.003266:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114256256:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765205.003277:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114256384:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765205.003307:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765205.003320:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765205/real 1775765205]  req@ffff9f1485745500 x1862022114256384/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765230 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765205.003334:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745500 x1862022114256384/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765230 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765205.003341:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765205.003345:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765205.003349:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114256384:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765205.003355:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114256512:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765205.003377:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765205.003388:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765205/real 1775765205]  req@ffff9f1485747480 x1862022114256512/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765230 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765205.003406:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747480 x1862022114256512/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765230 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765205.003415:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765205.003419:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765205.003423:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114256512:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775765205.003430:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114256640:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765205.003450:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765205.003462:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765205/real 1775765205]  req@ffff9f1485745f80 x1862022114256640/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765230 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765205.003475:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745f80 x1862022114256640/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765230 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765205.003484:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765205.003487:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765205.003490:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114256640:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775765205.003496:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114256768:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765205.003536:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765205.003551:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765205/real 1775765205]  req@ffff9f1485747100 x1862022114256768/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765230 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765205.003592:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747100 x1862022114256768/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765230 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765205.003608:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765205.003611:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765205.003615:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114256768:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765205.003638:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114256896:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765205.003663:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765205.003675:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765205/real 1775765205]  req@ffff9f1485744000 x1862022114256896/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765230 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765205.003685:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744000 x1862022114256896/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765230 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765205.003692:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765205.003695:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765205.003698:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114256896:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765205.003704:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114257024:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765205.003723:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765205.003733:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765205/real 1775765205]  req@ffff9f1485747800 x1862022114257024/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765230 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765205.003746:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747800 x1862022114257024/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765230 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765205.003754:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765205.003758:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765205.003762:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114257024:192.168.202.110@tcp:8:kworker.0
00010000:00010000:1.0:1775765209.354738:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775765210.122924:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114257152:192.168.202.110@tcp:250:kworker.0
00000800:00000100:1.0:1775765210.123059:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765210.123136:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765210/real 1775765210]  req@ffff9f1485745180 x1862022114257152/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765235 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765210.123229:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745180 x1862022114257152/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765235 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765210.123263:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765210.123294:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765210.123299:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114257152:192.168.202.110@tcp:250:kworker.0
00000100:00100000:1.0:1775765210.123308:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114257280:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765210.123332:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765210.123344:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765210/real 1775765210]  req@ffff9f1485747800 x1862022114257280/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765235 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765210.123355:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747800 x1862022114257280/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765235 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765210.123362:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765210.123364:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765210.123367:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114257280:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765210.123621:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114257408:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765210.123707:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765210.123736:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765210/real 1775765210]  req@ffff9f1485744000 x1862022114257408/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765235 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765210.123754:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744000 x1862022114257408/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765235 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765210.123764:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765210.123769:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765210.123772:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114257408:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765210.123779:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114257536:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765210.123796:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765210.123805:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765210/real 1775765210]  req@ffff9f1485747100 x1862022114257536/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765235 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765210.123814:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747100 x1862022114257536/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765235 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765210.123823:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765210.123827:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765210.123832:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114257536:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775765210.123840:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114257664:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765210.123865:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765210.123880:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765210/real 1775765210]  req@ffff9f1485745f80 x1862022114257664/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765235 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765210.123908:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745f80 x1862022114257664/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765235 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765210.123919:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765210.123923:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765210.123938:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114257664:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775765210.123946:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114257792:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765210.123974:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765210.123990:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765210/real 1775765210]  req@ffff9f1485747480 x1862022114257792/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765235 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765210.124001:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747480 x1862022114257792/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765235 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765210.124008:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765210.124011:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765210.124014:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114257792:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765210.124019:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114257920:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765210.124037:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765210.124045:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765210/real 1775765210]  req@ffff9f1485745500 x1862022114257920/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765235 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765210.124064:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745500 x1862022114257920/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765235 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765210.124076:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765210.124079:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765210.124081:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114257920:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765210.124086:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114258048:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765210.124115:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765210.124124:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765210/real 1775765210]  req@ffff9f1485745c00 x1862022114258048/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765235 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765210.124134:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745c00 x1862022114258048/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765235 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765210.124142:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765210.124146:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765210.124151:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114258048:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775765215.242839:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114258176:192.168.202.110@tcp:250:kworker.0
00000800:00000100:1.0:1775765215.242898:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765215.242937:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765215/real 1775765215]  req@ffff9f1485746300 x1862022114258176/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765240 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765215.242969:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746300 x1862022114258176/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765240 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765215.242991:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765215.243000:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765215.243008:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114258176:192.168.202.110@tcp:250:kworker.0
00000100:00100000:1.0:1775765215.243038:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114258304:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765215.243064:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765215.243078:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765215/real 1775765215]  req@ffff9f1485744700 x1862022114258304/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765240 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765215.243108:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744700 x1862022114258304/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765240 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765215.243128:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765215.243133:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765215.243140:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114258304:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765215.243147:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114258432:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765215.243162:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765215.243168:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765215/real 1775765215]  req@ffff9f1485744380 x1862022114258432/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765240 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765215.243176:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744380 x1862022114258432/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765240 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765215.243181:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765215.243183:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765215.243185:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114258432:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765215.243189:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114258560:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765215.243199:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765215.243205:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765215/real 1775765215]  req@ffff9f1485746a00 x1862022114258560/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765240 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765215.243213:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746a00 x1862022114258560/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765240 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765215.243217:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765215.243219:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765215.243221:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114258560:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775765215.243224:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114258688:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765215.243233:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765215.243238:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765215/real 1775765215]  req@ffff9f1485746680 x1862022114258688/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765240 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765215.243245:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746680 x1862022114258688/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765240 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765215.243250:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765215.243251:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765215.243253:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114258688:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775765215.243256:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114258816:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765215.243265:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765215.243270:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765215/real 1775765215]  req@ffff9f1485745880 x1862022114258816/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765240 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765215.243276:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745880 x1862022114258816/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765240 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765215.243285:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765215.243287:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765215.243289:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114258816:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765215.243299:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114258944:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765215.243309:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765215.243314:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765215/real 1775765215]  req@ffff9f1485744e00 x1862022114258944/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765240 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765215.243321:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744e00 x1862022114258944/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765240 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765215.243326:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765215.243328:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765215.243329:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114258944:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765215.243333:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114259072:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765215.243342:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765215.243347:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765215/real 1775765215]  req@ffff9f1485747b80 x1862022114259072/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765240 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765215.243354:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747b80 x1862022114259072/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765240 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765215.243358:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765215.243360:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765215.243363:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114259072:192.168.202.110@tcp:8:kworker.0
00010000:00010000:1.0:1775765216.522739:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775765220.362822:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114259200:192.168.202.110@tcp:250:kworker.0
00000800:00000100:1.0:1775765220.362883:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765220.362921:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765220/real 1775765220]  req@ffff9f14a04d3100 x1862022114259200/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765245 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765220.362942:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3100 x1862022114259200/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765245 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765220.362957:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765220.362970:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765220.362978:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114259200:192.168.202.110@tcp:250:kworker.0
00000100:00100000:1.0:1775765220.362996:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114259328:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765220.363024:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765220.363049:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765220/real 1775765220]  req@ffff9f14a04d3b80 x1862022114259328/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765245 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765220.363059:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3b80 x1862022114259328/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765245 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765220.363065:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765220.363067:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765220.363070:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114259328:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765220.363088:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114259456:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765220.363104:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765220.363110:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765220/real 1775765220]  req@ffff9f14a04d1f80 x1862022114259456/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765245 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765220.363118:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1f80 x1862022114259456/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765245 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765220.363123:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765220.363126:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765220.363128:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114259456:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765220.363131:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114259584:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765220.363141:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765220.363146:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765220/real 1775765220]  req@ffff9f14a04d0380 x1862022114259584/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765245 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765220.363155:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0380 x1862022114259584/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765245 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765220.363162:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765220.363166:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765220.363169:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114259584:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775765220.363175:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114259712:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765220.363194:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765220.363204:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765220/real 1775765220]  req@ffff9f14a04d0a80 x1862022114259712/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765245 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765220.363224:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0a80 x1862022114259712/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765245 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765220.363229:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765220.363231:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765220.363233:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114259712:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775765220.363237:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114259840:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765220.363276:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765220.363298:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765220/real 1775765220]  req@ffff9f14a04d2a00 x1862022114259840/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765245 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765220.363312:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d2a00 x1862022114259840/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765245 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765220.363320:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765220.363322:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765220.363324:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114259840:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765220.363329:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114259968:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765220.363344:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765220.363351:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765220/real 1775765220]  req@ffff9f14a04d0e00 x1862022114259968/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765245 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765220.363359:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0e00 x1862022114259968/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765245 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765220.363432:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765220.363435:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765220.363437:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114259968:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765220.363442:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114260096:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765220.363459:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765220.363467:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765220/real 1775765220]  req@ffff9f14a04d2300 x1862022114260096/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765245 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765220.363474:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d2300 x1862022114260096/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765245 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765220.363479:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765220.363481:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765220.363483:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114260096:192.168.202.110@tcp:8:kworker.0
00010000:00010000:1.0:1775765223.690830:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775765225.482979:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114260224:192.168.202.110@tcp:250:kworker.0
00000800:00000100:1.0:1775765225.483108:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765225.483188:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765225/real 1775765225]  req@ffff9f14852d1880 x1862022114260224/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765250 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765225.483277:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1880 x1862022114260224/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765250 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765225.483368:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765225.483386:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765225.483403:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114260224:192.168.202.110@tcp:250:kworker.0
00000100:00100000:1.0:1775765225.483435:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114260352:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765225.483500:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765225.483527:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765225/real 1775765225]  req@ffff9f14852d0e00 x1862022114260352/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765250 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765225.483551:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0e00 x1862022114260352/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765250 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765225.483567:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765225.483647:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765225.483660:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114260352:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765225.483735:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114260480:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765225.483801:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765225.483825:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765225/real 1775765225]  req@ffff9f14852d1c00 x1862022114260480/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765250 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765225.483848:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1c00 x1862022114260480/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765250 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765225.483863:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765225.483869:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765225.483875:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114260480:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765225.483886:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114260608:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765225.483917:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765225.483931:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765225/real 1775765225]  req@ffff9f14852d3480 x1862022114260608/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765250 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765225.483945:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3480 x1862022114260608/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765250 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765225.483954:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765225.483958:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765225.483962:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114260608:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775765225.483968:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114260736:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765225.483986:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765225.483995:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765225/real 1775765225]  req@ffff9f14852d2300 x1862022114260736/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765250 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765225.484007:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2300 x1862022114260736/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765250 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765225.484016:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765225.484019:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765225.484022:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114260736:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775765225.484028:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114260864:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765225.484044:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765225.484053:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765225/real 1775765225]  req@ffff9f14852d3100 x1862022114260864/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765250 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765225.484084:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3100 x1862022114260864/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765250 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765225.484095:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765225.484098:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765225.484102:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114260864:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765225.484108:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114260992:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765225.484127:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765225.484137:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765225/real 1775765225]  req@ffff9f14852d3b80 x1862022114260992/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765250 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765225.484150:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3b80 x1862022114260992/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765250 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765225.484159:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765225.484162:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765225.484166:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114260992:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765225.484172:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114261120:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765225.484189:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765225.484199:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765225/real 1775765225]  req@ffff9f14852d0000 x1862022114261120/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765250 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765225.484211:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0000 x1862022114261120/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765250 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765225.484220:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765225.484224:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765225.484228:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114261120:192.168.202.110@tcp:8:kworker.0
00000100:00080000:1.0:1775765230.602734:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765230.602751:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1775765230.602774:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765230.602790:0:8:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 2608
00000100:00080000:1.0:1775765230.602806:0:8:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765230.602816:0:8:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775765230.602869:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fed80 x1862022114261248/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765230.602911:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765230.602915:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775765230.602917:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765230.602922:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2608
00000100:00080000:1.0:1775765230.602926:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765230.602930:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765230.602943:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775765230.602963:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fca80 x1862022114261376/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765230.602981:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765230.602986:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775765230.602988:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765230.602995:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2608
00000100:00080000:1.0:1775765230.603001:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765230.603012:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765230.603024:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775765230.603043:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fc380 x1862022114261504/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765230.603058:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765230.603063:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775765230.603066:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765230.603072:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2608
00000100:00080000:1.0:1775765230.603099:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765230.603105:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765230.603123:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775765230.603143:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fea00 x1862022114261632/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765230.603159:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765230.603164:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1775765230.603166:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765230.603172:0:8:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2608
00000100:00080000:1.0:1775765230.603178:0:8:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765230.603193:0:8:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765230.603201:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775765230.603220:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff800 x1862022114261760/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765230.603236:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765230.603259:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775765230.603262:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765230.603267:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 2608
00000100:00080000:1.0:1775765230.603290:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765230.603300:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765230.603310:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775765230.603329:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff100 x1862022114261888/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765230.603340:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765230.603343:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775765230.603345:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765230.603354:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 2608
00000100:00080000:1.0:1775765230.603360:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765230.603364:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765230.603368:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775765230.603379:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fe300 x1862022114262016/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765230.603399:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765230.603404:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775765230.603408:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765230.603418:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 2608
00000100:00080000:1.0:1775765230.603426:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765230.603432:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765230.603448:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775765230.603464:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fce00 x1862022114262144/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00010000:00010000:1.0:1775765230.858755:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1775765235.722651:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765235.722661:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1775765235.722666:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765235.722674:0:8:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 2613
00000100:00080000:1.0:1775765235.722686:0:8:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765235.722693:0:8:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775765235.722744:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdc00 x1862022114262272/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765235.722771:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765235.722774:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775765235.722776:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765235.722788:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2613
00000100:00080000:1.0:1775765235.722791:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765235.722798:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765235.722808:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775765235.722820:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff480 x1862022114262400/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765235.722831:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765235.722839:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775765235.722844:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765235.722851:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2613
00000100:00080000:1.0:1775765235.722856:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765235.722861:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765235.722867:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775765235.722884:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fd880 x1862022114262528/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765235.722914:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765235.722919:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775765235.722922:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765235.722927:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2613
00000100:00080000:1.0:1775765235.722931:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765235.722935:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765235.722942:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775765235.722958:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdf80 x1862022114262656/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765235.722992:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765235.722995:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1775765235.722997:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765235.723006:0:8:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2613
00000100:00080000:1.0:1775765235.723011:0:8:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765235.723019:0:8:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765235.723027:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775765235.723042:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fc700 x1862022114262784/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765235.723063:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765235.723071:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775765235.723077:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765235.723086:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 2613
00000100:00080000:1.0:1775765235.723091:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765235.723099:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765235.723111:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775765235.723132:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ffb80 x1862022114262912/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765235.723152:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765235.723164:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775765235.723168:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765235.723179:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 2613
00000100:00080000:1.0:1775765235.723199:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765235.723208:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765235.723219:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775765235.723279:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdc00 x1862022114263040/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765235.723289:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765235.723296:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775765235.723301:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765235.723306:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 2613
00000100:00080000:1.0:1775765235.723312:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765235.723320:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765235.723328:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775765235.723341:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fce00 x1862022114263168/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00010000:00010000:1.0:1775765238.026652:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1775765240.842728:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765240.842742:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1775765240.842750:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765240.842766:0:8:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 2618
00000100:00080000:1.0:1775765240.842783:0:8:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765240.842793:0:8:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775765240.842854:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fe300 x1862022114263296/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765240.842896:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765240.842901:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775765240.842904:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765240.842910:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2618
00000100:00080000:1.0:1775765240.842915:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765240.842920:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765240.842935:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775765240.842955:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff100 x1862022114263424/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765240.842972:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765240.842978:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775765240.842980:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765240.842987:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2618
00000100:00080000:1.0:1775765240.843006:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765240.843012:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765240.843037:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775765240.843062:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff800 x1862022114263552/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765240.843090:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765240.843096:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775765240.843103:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765240.843111:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2618
00000100:00080000:1.0:1775765240.843120:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765240.843130:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765240.843142:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775765240.843161:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fea00 x1862022114263680/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765240.843225:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765240.843233:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1775765240.843237:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765240.843253:0:8:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2618
00000100:00080000:1.0:1775765240.843265:0:8:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765240.843286:0:8:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765240.843328:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775765240.843356:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fc380 x1862022114263808/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765240.843396:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765240.843402:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775765240.843405:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765240.843413:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 2618
00000100:00080000:1.0:1775765240.843420:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765240.843440:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765240.843451:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775765240.843475:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fca80 x1862022114263936/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765240.843508:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765240.843529:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775765240.843536:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765240.843549:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 2618
00000100:00080000:1.0:1775765240.843605:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765240.843616:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765240.843625:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775765240.843653:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fed80 x1862022114264064/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765240.843678:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765240.843692:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775765240.843697:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765240.843709:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 2618
00000100:00080000:1.0:1775765240.843715:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765240.843725:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765240.843736:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775765240.843756:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff800 x1862022114264192/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00010000:00010000:1.0:1775765242.122710:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00010000:00010000:1.0:1775765245.194817:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00010000:00010000:1.0:1775765249.290845:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00010000:00010000:1.0:1775765256.458740:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1775765261.322919:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765261.322933:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1775765261.322942:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765261.322958:0:8:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 2638
00000100:00080000:1.0:1775765261.322977:0:8:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765261.323009:0:8:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775765261.323091:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff100 x1862022114267392/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765261.323131:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765261.323138:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775765261.323142:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765261.323152:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2638
00000100:00080000:1.0:1775765261.323180:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765261.323195:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765261.323213:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775765261.323239:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fe300 x1862022114267520/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765261.323256:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765261.323260:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775765261.323262:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765261.323275:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2638
00000100:00080000:1.0:1775765261.323288:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765261.323297:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765261.323307:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775765261.323329:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fce00 x1862022114267648/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765261.323342:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765261.323347:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775765261.323349:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765261.323361:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2638
00000100:00080000:1.0:1775765261.323370:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765261.323379:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765261.323391:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775765261.323430:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdc00 x1862022114267776/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765261.323505:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765261.323518:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1775765261.323530:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765261.323543:0:8:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2638
00000100:00080000:1.0:1775765261.323551:0:8:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765261.323649:0:8:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765261.323658:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775765261.323695:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ffb80 x1862022114267904/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765261.323731:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765261.323746:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775765261.323754:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765261.323766:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 2638
00000100:00080000:1.0:1775765261.323776:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765261.323783:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765261.323792:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775765261.323812:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fc700 x1862022114268032/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765261.323839:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765261.323856:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775765261.323865:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765261.323881:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 2638
00000100:00080000:1.0:1775765261.323887:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765261.323901:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765261.323917:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775765261.323950:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdf80 x1862022114268160/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765261.324014:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765261.324022:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775765261.324026:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765261.324055:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 2638
00000100:00080000:1.0:1775765261.324061:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765261.324068:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765261.324078:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775765261.324097:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fd880 x1862022114268288/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00010000:00010000:1.0:1775765263.626905:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00010000:00010000:1.0:1775765270.794732:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00010000:00010000:1.0:1775765277.962731:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00010000:00010000:1.0:1775765285.130708:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1775765286.922740:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765286.922782:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1775765286.922791:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765286.922809:0:8:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 2664
00000100:00080000:1.0:1775765286.922827:0:8:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765286.922838:0:8:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775765286.922900:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff480 x1862022114272512/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765286.922946:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765286.922951:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775765286.922955:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765286.922964:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2664
00000100:00080000:1.0:1775765286.922970:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765286.922976:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765286.922994:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775765286.923024:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fd880 x1862022114272640/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765286.923041:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765286.923046:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775765286.923049:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765286.923055:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2664
00000100:00080000:1.0:1775765286.923063:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765286.923068:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765286.923083:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775765286.923100:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdf80 x1862022114272768/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765286.923114:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765286.923120:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775765286.923122:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765286.923130:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2664
00000100:00080000:1.0:1775765286.923139:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765286.923153:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765286.923169:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775765286.923189:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fc700 x1862022114272896/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765286.923223:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765286.923232:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1775765286.923236:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765286.923256:0:8:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2664
00000100:00080000:1.0:1775765286.923261:0:8:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765286.923268:0:8:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765286.923275:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775765286.923293:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ffb80 x1862022114273024/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765286.923308:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765286.923313:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775765286.923316:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765286.923324:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 2664
00000100:00080000:1.0:1775765286.923333:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765286.923350:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765286.923358:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775765286.923383:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdc00 x1862022114273152/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765286.923412:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765286.923423:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775765286.923429:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765286.923436:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 2664
00000100:00080000:1.0:1775765286.923447:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765286.923462:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765286.923472:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775765286.923491:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fce00 x1862022114273280/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765286.923515:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765286.923520:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775765286.923522:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765286.923530:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 2664
00000100:00080000:1.0:1775765286.923541:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765286.923642:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765286.923651:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775765286.923690:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fe300 x1862022114273408/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765292.042628:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765292.042638:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1775765292.042641:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765292.042649:0:8:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 2669
00000100:00080000:1.0:1775765292.042658:0:8:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765292.042663:0:8:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775765292.042714:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff100 x1862022114273536/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765292.042741:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765292.042746:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775765292.042748:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765292.042752:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2669
00000100:00080000:1.0:1775765292.042756:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765292.042761:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765292.042771:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775765292.042787:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff800 x1862022114273664/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765292.042800:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765292.042803:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775765292.042804:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765292.042810:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2669
00000100:00080000:1.0:1775765292.042827:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765292.042831:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765292.042837:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775765292.042851:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fed80 x1862022114273792/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765292.042860:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765292.042863:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775765292.042865:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765292.042871:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2669
00000100:00080000:1.0:1775765292.042878:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765292.042885:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765292.042896:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775765292.042909:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fca80 x1862022114273920/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765292.042926:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765292.042930:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1775765292.042933:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765292.042942:0:8:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2669
00000100:00080000:1.0:1775765292.042946:0:8:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765292.042949:0:8:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765292.042954:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775765292.042965:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fc380 x1862022114274048/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765292.042991:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765292.042994:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775765292.042999:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765292.043007:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 2669
00000100:00080000:1.0:1775765292.043017:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765292.043025:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765292.043036:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775765292.043050:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fea00 x1862022114274176/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765292.043082:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765292.043090:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775765292.043094:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765292.043103:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 2669
00000100:00080000:1.0:1775765292.043110:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765292.043119:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765292.043124:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775765292.043138:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff100 x1862022114274304/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765292.043156:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765292.043161:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775765292.043162:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765292.043170:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 2669
00000100:00080000:1.0:1775765292.043177:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765292.043185:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765292.043194:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775765292.043222:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fe300 x1862022114274432/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00010000:00010000:1.0:1775765292.298736:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775765297.162966:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114274560:192.168.202.110@tcp:250:kworker.0
00000800:00000100:1.0:1775765297.163147:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765297.163238:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765297/real 1775765297]  req@ffff9f14852d3800 x1862022114274560/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765322 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765297.163280:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3800 x1862022114274560/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765322 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765297.163336:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765297.163355:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765297.163373:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114274560:192.168.202.110@tcp:250:kworker.0
00000100:00100000:1.0:1775765297.163428:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114274688:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765297.163507:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765297.163531:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765297/real 1775765297]  req@ffff9f14852d0380 x1862022114274688/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765322 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765297.163640:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0380 x1862022114274688/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765322 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765297.163695:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765297.163704:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765297.163716:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114274688:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765297.163753:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114274816:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765297.163825:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765297.163856:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765297/real 1775765297]  req@ffff9f14852d0000 x1862022114274816/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765322 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765297.163886:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0000 x1862022114274816/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765322 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765297.163909:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765297.163917:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765297.163923:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114274816:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765297.163934:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114274944:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765297.163961:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765297.163979:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765297/real 1775765297]  req@ffff9f14852d3b80 x1862022114274944/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765322 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765297.164004:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3b80 x1862022114274944/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765322 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765297.164019:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765297.164026:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765297.164032:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114274944:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775765297.164065:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114275072:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765297.164107:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765297.164124:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765297/real 1775765297]  req@ffff9f14852d3100 x1862022114275072/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765322 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765297.164140:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3100 x1862022114275072/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765322 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765297.164150:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765297.164155:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765297.164159:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114275072:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775765297.164167:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114275200:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765297.164189:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765297.164201:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765297/real 1775765297]  req@ffff9f14852d2300 x1862022114275200/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765322 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765297.164216:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2300 x1862022114275200/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765322 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765297.164225:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765297.164229:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765297.164232:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114275200:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765297.164239:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114275328:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765297.164256:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765297.164266:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765297/real 1775765297]  req@ffff9f14852d3480 x1862022114275328/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765322 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765297.164280:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3480 x1862022114275328/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765322 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765297.164289:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765297.164293:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765297.164296:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114275328:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765297.164303:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114275456:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765297.164330:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765297.164339:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765297/real 1775765297]  req@ffff9f14852d1c00 x1862022114275456/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765322 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765297.164352:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1c00 x1862022114275456/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765322 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765297.164370:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765297.164374:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765297.164378:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114275456:192.168.202.110@tcp:8:kworker.0
00010000:00010000:1.0:1775765299.466779:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775765302.282780:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114275584:192.168.202.110@tcp:250:kworker.0
00000800:00000100:1.0:1775765302.282843:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765302.282885:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765302/real 1775765302]  req@ffff9f1485744700 x1862022114275584/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765327 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765302.282923:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744700 x1862022114275584/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765327 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765302.282939:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765302.282946:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765302.282952:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114275584:192.168.202.110@tcp:250:kworker.0
00000100:00100000:1.0:1775765302.282978:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114275712:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765302.283008:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765302.283024:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765302/real 1775765302]  req@ffff9f1485745c00 x1862022114275712/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765327 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765302.283052:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745c00 x1862022114275712/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765327 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765302.283065:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765302.283074:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765302.283080:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114275712:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765302.283086:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114275840:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765302.283101:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765302.283109:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765302/real 1775765302]  req@ffff9f1485745500 x1862022114275840/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765327 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765302.283118:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745500 x1862022114275840/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765327 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765302.283122:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765302.283124:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765302.283126:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114275840:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765302.283130:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114275968:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765302.283141:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765302.283146:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765302/real 1775765302]  req@ffff9f1485747480 x1862022114275968/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765327 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765302.283154:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747480 x1862022114275968/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765327 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765302.283159:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765302.283160:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765302.283162:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114275968:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775765302.283165:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114276096:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765302.283174:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765302.283180:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765302/real 1775765302]  req@ffff9f1485745f80 x1862022114276096/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765327 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765302.283187:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745f80 x1862022114276096/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765327 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765302.283192:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765302.283194:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765302.283196:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114276096:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775765302.283206:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114276224:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765302.283215:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765302.283221:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765302/real 1775765302]  req@ffff9f1485747100 x1862022114276224/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765327 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765302.283228:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747100 x1862022114276224/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765327 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765302.283233:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765302.283235:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765302.283237:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114276224:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765302.283241:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114276352:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765302.283250:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765302.283255:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765302/real 1775765302]  req@ffff9f1485744000 x1862022114276352/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765327 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765302.283262:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744000 x1862022114276352/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765327 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765302.283267:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765302.283269:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765302.283271:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114276352:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765302.283274:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114276480:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765302.283283:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765302.283292:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765302/real 1775765302]  req@ffff9f1485747800 x1862022114276480/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765327 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765302.283302:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747800 x1862022114276480/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765327 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765302.283310:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765302.283313:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765302.283316:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114276480:192.168.202.110@tcp:8:kworker.0
00010000:00010000:1.0:1775765306.634795:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775765307.403025:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114276608:192.168.202.110@tcp:250:kworker.0
00000800:00000100:1.0:1775765307.403153:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765307.403245:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765307/real 1775765307]  req@ffff9f14852d0e00 x1862022114276608/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765332 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765307.403335:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0e00 x1862022114276608/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765332 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765307.403368:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765307.403389:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765307.403408:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114276608:192.168.202.110@tcp:250:kworker.0
00000100:00100000:1.0:1775765307.403436:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114276736:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765307.403493:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765307.403536:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765307/real 1775765307]  req@ffff9f14852d1880 x1862022114276736/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765332 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765307.403641:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1880 x1862022114276736/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765332 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765307.403655:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765307.403661:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765307.403666:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114276736:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765307.403715:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114276864:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765307.403753:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765307.403766:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765307/real 1775765307]  req@ffff9f14852d1c00 x1862022114276864/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765332 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765307.403778:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1c00 x1862022114276864/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765332 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765307.403787:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765307.403791:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765307.403794:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114276864:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765307.403801:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114276992:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765307.403821:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765307.403832:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765307/real 1775765307]  req@ffff9f14852d3480 x1862022114276992/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765332 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765307.403845:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3480 x1862022114276992/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765332 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765307.403853:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765307.403857:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765307.403861:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114276992:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775765307.403867:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114277120:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765307.403889:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765307.403900:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765307/real 1775765307]  req@ffff9f14852d2300 x1862022114277120/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765332 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765307.403913:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2300 x1862022114277120/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765332 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765307.403922:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765307.403926:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765307.403929:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114277120:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775765307.403936:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114277248:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765307.403957:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765307.403967:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765307/real 1775765307]  req@ffff9f14852d3100 x1862022114277248/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765332 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765307.403979:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3100 x1862022114277248/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765332 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765307.403988:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765307.403992:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765307.403995:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114277248:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765307.404002:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114277376:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765307.404022:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765307.404032:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765307/real 1775765307]  req@ffff9f14852d3b80 x1862022114277376/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765332 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765307.404044:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3b80 x1862022114277376/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765332 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765307.404053:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765307.404057:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765307.404060:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114277376:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765307.404067:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114277504:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765307.404106:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765307.404117:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765307/real 1775765307]  req@ffff9f14852d0000 x1862022114277504/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765332 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765307.404131:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0000 x1862022114277504/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765332 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765307.404155:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765307.404158:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765307.404163:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114277504:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775765312.522905:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114277632:192.168.202.110@tcp:250:kworker.0
00000800:00000100:1.0:1775765312.522975:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765312.523023:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765312/real 1775765312]  req@ffff9f1485745180 x1862022114277632/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765337 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765312.523079:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745180 x1862022114277632/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765337 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765312.523097:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765312.523103:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765312.523109:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114277632:192.168.202.110@tcp:250:kworker.0
00000100:00100000:1.0:1775765312.523121:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114277760:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765312.523170:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765312.523189:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765312/real 1775765312]  req@ffff9f1485747800 x1862022114277760/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765337 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765312.523202:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747800 x1862022114277760/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765337 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765312.523212:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765312.523216:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765312.523224:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114277760:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765312.523267:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114277888:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765312.523307:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765312.523322:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765312/real 1775765312]  req@ffff9f1485744000 x1862022114277888/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765337 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765312.523332:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744000 x1862022114277888/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765337 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765312.523339:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765312.523342:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765312.523345:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114277888:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765312.523350:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114278016:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765312.523367:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765312.523373:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765312/real 1775765312]  req@ffff9f1485747100 x1862022114278016/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765337 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765312.523383:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747100 x1862022114278016/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765337 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765312.523388:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765312.523390:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765312.523393:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114278016:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775765312.523397:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114278144:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765312.523408:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765312.523414:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765312/real 1775765312]  req@ffff9f1485745f80 x1862022114278144/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765337 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765312.523423:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745f80 x1862022114278144/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765337 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765312.523430:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765312.523432:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765312.523435:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114278144:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775765312.523456:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114278272:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765312.523471:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765312.523479:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765312/real 1775765312]  req@ffff9f1485747480 x1862022114278272/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765337 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765312.523489:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747480 x1862022114278272/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765337 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765312.523496:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765312.523499:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765312.523501:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114278272:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765312.523602:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114278400:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765312.523622:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765312.523631:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765312/real 1775765312]  req@ffff9f1485745500 x1862022114278400/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765337 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765312.523640:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745500 x1862022114278400/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765337 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765312.523646:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765312.523649:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765312.523653:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114278400:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765312.523668:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114278528:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765312.523681:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765312.523688:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765312/real 1775765312]  req@ffff9f1485745c00 x1862022114278528/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765337 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765312.523698:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745c00 x1862022114278528/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765337 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765312.523703:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765312.523705:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765312.523708:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114278528:192.168.202.110@tcp:8:kworker.0
00010000:00010000:1.0:1775765313.802718:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775765317.643026:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114278656:192.168.202.110@tcp:250:kworker.0
00000800:00000100:1.0:1775765317.643135:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765317.643197:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765317/real 1775765317]  req@ffff9f1485744700 x1862022114278656/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765342 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765317.643263:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744700 x1862022114278656/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765342 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765317.643277:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765317.643287:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765317.643297:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114278656:192.168.202.110@tcp:250:kworker.0
00000100:00100000:1.0:1775765317.643353:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114278784:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765317.643394:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765317.643426:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765317/real 1775765317]  req@ffff9f1485744380 x1862022114278784/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765342 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765317.643453:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744380 x1862022114278784/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765342 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765317.643534:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765317.643546:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765317.643573:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114278784:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765317.643668:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114278912:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765317.643722:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765317.643739:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765317/real 1775765317]  req@ffff9f1485746a00 x1862022114278912/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765342 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765317.643770:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746a00 x1862022114278912/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765342 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765317.643779:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765317.643783:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765317.643786:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114278912:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765317.643792:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114279040:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765317.643811:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765317.643819:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765317/real 1775765317]  req@ffff9f1485746300 x1862022114279040/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765342 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765317.643838:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746300 x1862022114279040/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765342 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765317.643845:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765317.643848:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765317.643851:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114279040:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775765317.643856:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114279168:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765317.643870:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765317.643878:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765317/real 1775765317]  req@ffff9f1485747b80 x1862022114279168/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765342 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765317.643889:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747b80 x1862022114279168/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765342 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765317.643897:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765317.643899:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765317.643902:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114279168:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775765317.643919:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114279296:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765317.643934:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765317.643942:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765317/real 1775765317]  req@ffff9f1485744e00 x1862022114279296/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765342 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765317.643952:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744e00 x1862022114279296/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765342 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765317.643966:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765317.643969:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765317.643972:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114279296:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765317.643977:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114279424:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765317.643990:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765317.643998:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765317/real 1775765317]  req@ffff9f1485746680 x1862022114279424/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765342 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765317.644009:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746680 x1862022114279424/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765342 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765317.644017:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765317.644020:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765317.644022:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114279424:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765317.644027:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114279552:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765317.644040:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765317.644048:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765317/real 1775765317]  req@ffff9f1485745880 x1862022114279552/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765342 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765317.644058:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745880 x1862022114279552/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765342 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765317.644065:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765317.644069:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765317.644072:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114279552:192.168.202.110@tcp:8:kworker.0
00010000:00010000:1.0:1775765320.970822:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775765322.762789:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114279680:192.168.202.110@tcp:250:kworker.0
00000800:00000100:1.0:1775765322.762847:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765322.762884:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765322/real 1775765322]  req@ffff9f1485744a80 x1862022114279680/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765347 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765322.762901:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744a80 x1862022114279680/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765347 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765322.762913:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765322.762917:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765322.762921:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114279680:192.168.202.110@tcp:250:kworker.0
00000100:00100000:1.0:1775765322.762942:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114279808:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765322.762963:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765322.762975:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765322/real 1775765322]  req@ffff9f1485745880 x1862022114279808/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765347 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765322.762988:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745880 x1862022114279808/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765347 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765322.762997:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765322.763001:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765322.763004:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114279808:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765322.763010:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114279936:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765322.763029:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765322.763051:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765322/real 1775765322]  req@ffff9f1485746680 x1862022114279936/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765347 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765322.763065:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746680 x1862022114279936/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765347 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765322.763071:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765322.763073:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765322.763075:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114279936:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765322.763080:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114280064:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765322.763095:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765322.763102:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765322/real 1775765322]  req@ffff9f1485744e00 x1862022114280064/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765347 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765322.763109:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744e00 x1862022114280064/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765347 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765322.763114:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765322.763116:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765322.763118:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114280064:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775765322.763122:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114280192:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765322.763131:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765322.763136:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765322/real 1775765322]  req@ffff9f1485747b80 x1862022114280192/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765347 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765322.763147:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747b80 x1862022114280192/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765347 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765322.763152:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765322.763154:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765322.763155:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114280192:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775765322.763167:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114280320:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765322.763176:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765322.763181:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765322/real 1775765322]  req@ffff9f1485746300 x1862022114280320/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765347 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765322.763188:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746300 x1862022114280320/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765347 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765322.763193:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765322.763195:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765322.763197:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114280320:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765322.763200:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114280448:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765322.763217:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765322.763223:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765322/real 1775765322]  req@ffff9f1485746a00 x1862022114280448/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765347 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765322.763230:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746a00 x1862022114280448/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765347 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765322.763239:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765322.763241:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765322.763244:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114280448:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765322.763247:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114280576:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765322.763256:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765322.763261:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765322/real 1775765322]  req@ffff9f1485744380 x1862022114280576/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765347 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765322.763268:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744380 x1862022114280576/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765347 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765322.763273:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765322.763274:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765322.763276:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114280576:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775765325.834818:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:1.0:1775765326.858654:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:1.0:1775765327.882656:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:1.0:1775765327.882773:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114280704:192.168.202.110@tcp:250:kworker.0
00000800:00000100:1.0:1775765327.882854:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765327.882908:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765327/real 1775765327]  req@ffff9f1485744700 x1862022114280704/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765352 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765327.882949:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744700 x1862022114280704/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765352 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765327.883022:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765327.883033:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765327.883052:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114280704:192.168.202.110@tcp:250:kworker.0
00000100:00100000:1.0:1775765327.883091:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114280832:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765327.883127:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765327.883143:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765327/real 1775765327]  req@ffff9f1485745c00 x1862022114280832/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765352 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765327.883155:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745c00 x1862022114280832/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765352 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765327.883162:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765327.883165:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765327.883167:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114280832:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765327.883172:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114280960:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765327.883184:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765327.883190:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765327/real 1775765327]  req@ffff9f1485745500 x1862022114280960/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765352 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765327.883197:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745500 x1862022114280960/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765352 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765327.883202:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765327.883204:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765327.883206:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114280960:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765327.883211:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114281088:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765327.883222:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765327.883228:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765327/real 1775765327]  req@ffff9f1485747480 x1862022114281088/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765352 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765327.883236:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747480 x1862022114281088/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765352 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765327.883241:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765327.883243:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765327.883244:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114281088:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775765327.883248:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114281216:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765327.883257:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765327.883262:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765327/real 1775765327]  req@ffff9f1485745f80 x1862022114281216/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765352 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765327.883269:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745f80 x1862022114281216/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765352 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765327.883273:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765327.883275:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765327.883276:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114281216:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775765327.883280:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114281344:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765327.883290:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765327.883295:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765327/real 1775765327]  req@ffff9f1485747100 x1862022114281344/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765352 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765327.883303:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747100 x1862022114281344/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765352 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765327.883308:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765327.883310:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765327.883311:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114281344:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765327.883324:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114281472:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765327.883333:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765327.883339:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765327/real 1775765327]  req@ffff9f1485744000 x1862022114281472/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765352 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765327.883345:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744000 x1862022114281472/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765352 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765327.883350:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765327.883353:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765327.883354:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114281472:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765327.883358:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114281600:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765327.883368:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765327.883401:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765327/real 1775765327]  req@ffff9f1485747800 x1862022114281600/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765352 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765327.883409:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747800 x1862022114281600/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765352 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765327.883425:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765327.883429:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765327.883432:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114281600:192.168.202.110@tcp:8:kworker.0
00010000:00010000:1.0:1775765328.138745:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775765328.906728:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:1.0:1775765329.930628:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:1.0:1775765330.954705:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:1.0:1775765331.978723:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:1.0:1775765333.002713:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775765335.306937:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775765340.170740:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:1.0:1775765341.194637:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:1.0:1775765342.218636:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775765342.474845:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775765343.242652:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:1.0:1775765344.266710:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:1.0:1775765345.290755:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:1.0:1775765346.314770:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:1.0:1775765347.338626:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:1.0:1775765348.362640:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:1.0:1775765349.386637:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775765349.642769:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775765350.410606:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:1.0:1775765351.434720:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:1.0:1775765352.458689:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:1.0:1775765353.482685:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:1.0:1775765353.482964:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114285824:192.168.202.110@tcp:250:kworker.0
00000800:00000100:1.0:1775765353.483077:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765353.483189:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765353/real 1775765353]  req@ffff9f14852d0380 x1862022114285824/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765378 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765353.483236:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0380 x1862022114285824/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765378 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765353.483267:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765353.483277:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765353.483285:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114285824:192.168.202.110@tcp:250:kworker.0
00000100:00100000:1.0:1775765353.483325:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114285952:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765353.483369:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765353.483399:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765353/real 1775765353]  req@ffff9f14852d3800 x1862022114285952/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765378 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765353.483423:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3800 x1862022114285952/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765378 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765353.483435:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765353.483439:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765353.483444:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114285952:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765353.483453:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114286080:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765353.483496:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765353.483519:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765353/real 1775765353]  req@ffff9f14852d2d80 x1862022114286080/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765378 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765353.483548:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2d80 x1862022114286080/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765378 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765353.483604:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765353.483612:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765353.483626:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114286080:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765353.483641:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114286208:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765353.483678:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765353.483702:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765353/real 1775765353]  req@ffff9f14852d0700 x1862022114286208/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765378 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765353.483758:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0700 x1862022114286208/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765378 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765353.483785:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765353.483800:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765353.483816:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114286208:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775765353.483838:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114286336:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765353.483879:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765353.483896:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765353/real 1775765353]  req@ffff9f14852d0a80 x1862022114286336/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765378 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765353.483911:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0a80 x1862022114286336/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765378 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765353.483920:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765353.483924:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765353.483929:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114286336:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775765353.483953:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114286464:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765353.483972:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765353.483984:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765353/real 1775765353]  req@ffff9f14852d1500 x1862022114286464/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765378 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765353.483998:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1500 x1862022114286464/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765378 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765353.484007:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765353.484011:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765353.484015:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114286464:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765353.484021:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114286592:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765353.484043:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765353.484053:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765353/real 1775765353]  req@ffff9f14852d2680 x1862022114286592/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765378 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765353.484067:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2680 x1862022114286592/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765378 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765353.484076:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765353.484079:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765353.484082:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114286592:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765353.484089:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114286720:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765353.484106:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765353.484116:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765353/real 1775765353]  req@ffff9f14852d2a00 x1862022114286720/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765378 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765353.484130:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2a00 x1862022114286720/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765378 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765353.484171:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765353.484176:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765353.484190:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114286720:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775765354.506649:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:1.0:1775765355.530785:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:1.0:1775765356.554670:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775765356.810794:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775765357.578656:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:1.0:1775765358.602651:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:1.0:1775765358.602780:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114286848:192.168.202.110@tcp:250:kworker.0
00000800:00000100:1.0:1775765358.602886:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765358.602944:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765358/real 1775765358]  req@ffff9f14852d1180 x1862022114286848/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765383 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765358.603009:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1180 x1862022114286848/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765383 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765358.603045:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765358.603056:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765358.603066:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114286848:192.168.202.110@tcp:250:kworker.0
00000100:00100000:1.0:1775765358.603117:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114286976:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765358.603156:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765358.603173:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765358/real 1775765358]  req@ffff9f14852d1f80 x1862022114286976/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765383 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765358.603191:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1f80 x1862022114286976/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765383 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765358.603198:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765358.603202:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765358.603204:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114286976:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765358.603209:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114287104:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765358.603222:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765358.603228:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765358/real 1775765358]  req@ffff9f14852d2a00 x1862022114287104/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765383 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765358.603236:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2a00 x1862022114287104/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765383 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765358.603241:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765358.603244:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765358.603246:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114287104:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765358.603249:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114287232:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765358.603260:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765358.603265:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765358/real 1775765358]  req@ffff9f14852d2680 x1862022114287232/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765383 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765358.603278:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2680 x1862022114287232/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765383 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765358.603283:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765358.603285:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765358.603286:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114287232:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775765358.603290:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114287360:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765358.603299:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765358.603305:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765358/real 1775765358]  req@ffff9f14852d1500 x1862022114287360/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765383 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765358.603311:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1500 x1862022114287360/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765383 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765358.603316:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765358.603318:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765358.603320:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114287360:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775765358.603324:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114287488:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765358.603333:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765358.603339:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765358/real 1775765358]  req@ffff9f14852d0a80 x1862022114287488/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765383 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765358.603345:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0a80 x1862022114287488/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765383 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765358.603354:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765358.603357:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765358.603359:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114287488:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765358.603362:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114287616:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765358.603372:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765358.603377:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765358/real 1775765358]  req@ffff9f14852d0700 x1862022114287616/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765383 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765358.603383:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0700 x1862022114287616/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765383 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765358.603388:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765358.603390:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765358.603391:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114287616:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765358.603406:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114287744:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765358.603417:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765358.603423:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765358/real 1775765358]  req@ffff9f14852d2d80 x1862022114287744/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765383 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765358.603430:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2d80 x1862022114287744/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765383 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765358.603435:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765358.603436:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765358.603439:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114287744:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775765359.626739:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:1.0:1775765360.650634:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:1.0:1775765361.674652:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:1.0:1775765362.698660:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:1.0:1775765363.722681:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:1.0:1775765363.722833:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114287872:192.168.202.110@tcp:250:kworker.0
00000800:00000100:1.0:1775765363.722930:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765363.722993:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765363/real 1775765363]  req@ffff9f14852d3800 x1862022114287872/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765388 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765363.723057:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3800 x1862022114287872/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765388 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765363.723073:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765363.723082:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765363.723089:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114287872:192.168.202.110@tcp:250:kworker.0
00000100:00100000:1.0:1775765363.723123:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114288000:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765363.723149:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765363.723173:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765363/real 1775765363]  req@ffff9f14852d0380 x1862022114288000/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765388 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765363.723186:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0380 x1862022114288000/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765388 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765363.723193:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765363.723195:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765363.723200:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114288000:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765363.723205:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114288128:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765363.723218:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765363.723223:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765363/real 1775765363]  req@ffff9f14852d0000 x1862022114288128/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765388 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765363.723231:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0000 x1862022114288128/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765388 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765363.723236:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765363.723238:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765363.723240:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114288128:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765363.723244:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114288256:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765363.723256:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765363.723261:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765363/real 1775765363]  req@ffff9f14852d3b80 x1862022114288256/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765388 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765363.723269:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3b80 x1862022114288256/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765388 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765363.723274:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765363.723276:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765363.723278:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114288256:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775765363.723281:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114288384:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765363.723290:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765363.723296:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765363/real 1775765363]  req@ffff9f14852d3100 x1862022114288384/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765388 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765363.723303:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3100 x1862022114288384/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765388 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765363.723308:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765363.723310:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765363.723312:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114288384:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775765363.723315:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114288512:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765363.723335:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765363.723340:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765363/real 1775765363]  req@ffff9f14852d2300 x1862022114288512/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765388 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765363.723347:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2300 x1862022114288512/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765388 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765363.723352:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765363.723354:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765363.723356:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114288512:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765363.723359:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114288640:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765363.723379:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765363.723385:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765363/real 1775765363]  req@ffff9f14852d3480 x1862022114288640/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765388 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765363.723391:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3480 x1862022114288640/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765388 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765363.723397:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765363.723399:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765363.723402:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114288640:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765363.723416:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114288768:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765363.723426:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765363.723431:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765363/real 1775765363]  req@ffff9f14852d1c00 x1862022114288768/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765388 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765363.723438:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1c00 x1862022114288768/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765388 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765363.723444:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765363.723445:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765363.723447:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114288768:192.168.202.110@tcp:8:kworker.0
00010000:00010000:1.0:1775765363.978743:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775765364.746654:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:1.0:1775765365.770685:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:1.0:1775765366.794655:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:1.0:1775765367.818750:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:1.0:1775765368.842692:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:1.0:1775765368.842904:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114288896:192.168.202.110@tcp:250:kworker.0
00000800:00000100:1.0:1775765368.843018:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765368.843088:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765368/real 1775765368]  req@ffff9f14a04d3100 x1862022114288896/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765393 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765368.843199:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3100 x1862022114288896/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765393 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765368.843257:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765368.843269:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765368.843287:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114288896:192.168.202.110@tcp:250:kworker.0
00000100:00100000:1.0:1775765368.843336:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114289024:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765368.843387:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765368.843406:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765368/real 1775765368]  req@ffff9f14a04d3b80 x1862022114289024/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765393 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765368.843421:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3b80 x1862022114289024/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765393 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765368.843433:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765368.843439:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765368.843445:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114289024:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765368.843452:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114289152:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765368.843481:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765368.843492:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765368/real 1775765368]  req@ffff9f14a04d1f80 x1862022114289152/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765393 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765368.843503:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1f80 x1862022114289152/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765393 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765368.843517:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765368.843521:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765368.843526:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114289152:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765368.843533:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114289280:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765368.843592:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765368.843616:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765368/real 1775765368]  req@ffff9f14a04d0380 x1862022114289280/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765393 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765368.843642:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0380 x1862022114289280/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765393 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765368.843653:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765368.843657:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765368.843660:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114289280:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775765368.843670:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114289408:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765368.843691:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765368.843708:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765368/real 1775765368]  req@ffff9f14a04d0a80 x1862022114289408/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765393 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765368.843735:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0a80 x1862022114289408/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765393 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765368.843744:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765368.843748:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765368.843753:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114289408:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775765368.843764:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114289536:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765368.843785:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765368.843814:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765368/real 1775765368]  req@ffff9f14a04d2a00 x1862022114289536/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765393 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765368.843827:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d2a00 x1862022114289536/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765393 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765368.843837:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765368.843843:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765368.843849:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114289536:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765368.843858:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114289664:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765368.843890:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765368.843903:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765368/real 1775765368]  req@ffff9f14a04d0e00 x1862022114289664/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765393 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765368.843916:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0e00 x1862022114289664/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765393 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765368.843938:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765368.843943:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765368.843950:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114289664:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765368.844007:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114289792:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765368.844044:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765368.844071:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765368/real 1775765368]  req@ffff9f14a04d2300 x1862022114289792/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765393 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765368.844083:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d2300 x1862022114289792/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765393 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765368.844090:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765368.844094:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765368.844097:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114289792:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775765369.866640:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:1.0:1775765370.890639:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775765371.146804:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775765371.914648:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:1.0:1775765372.938693:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:1.0:1775765373.962760:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:1.0:1775765374.986715:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:1.0:1775765376.010675:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:1.0:1775765377.034721:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:1.0:1775765378.058753:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775765378.314830:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775765379.082732:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:1.0:1775765380.106651:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:1.0:1775765381.130724:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:1.0:1775765382.154810:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:1.0:1775765383.178640:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:1.0:1775765384.202694:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:1.0:1775765385.226634:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775765385.482804:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775765386.250690:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:1.0:1775765387.274670:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:1.0:1775765388.298687:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:1.0:1775765389.322764:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:1.0:1775765390.346682:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:1.0:1775765391.370728:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:1.0:1775765392.394658:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775765392.650689:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775765393.418629:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:1.0:1775765394.442713:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:1.0:1775765394.443041:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114294016:192.168.202.110@tcp:250:kworker.0
00000800:00000100:1.0:1775765394.443156:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765394.443244:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765394/real 1775765394]  req@ffff9f14852d1880 x1862022114294016/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765419 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765394.443303:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1880 x1862022114294016/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765419 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765394.443340:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765394.443358:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765394.443368:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114294016:192.168.202.110@tcp:250:kworker.0
00000100:00100000:1.0:1775765394.443409:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114294144:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765394.443468:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765394.443500:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765394/real 1775765394]  req@ffff9f14852d0e00 x1862022114294144/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765419 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765394.443541:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0e00 x1862022114294144/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765419 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765394.443605:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765394.443621:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765394.443638:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114294144:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765394.443652:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114294272:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765394.443685:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765394.443701:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765394/real 1775765394]  req@ffff9f14852d1c00 x1862022114294272/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765419 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765394.443716:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1c00 x1862022114294272/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765419 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765394.443726:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765394.443730:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765394.443734:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114294272:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765394.443741:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114294400:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765394.443804:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765394.443826:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765394/real 1775765394]  req@ffff9f14852d3480 x1862022114294400/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765419 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765394.443849:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3480 x1862022114294400/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765419 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765394.443865:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765394.443871:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765394.443878:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114294400:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775765394.443890:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114294528:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765394.443928:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765394.443945:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765394/real 1775765394]  req@ffff9f14852d2300 x1862022114294528/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765419 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765394.443960:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2300 x1862022114294528/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765419 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765394.443970:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765394.443973:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765394.443977:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114294528:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775765394.443984:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114294656:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765394.444004:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765394.444017:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765394/real 1775765394]  req@ffff9f14852d3100 x1862022114294656/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765419 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765394.444042:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3100 x1862022114294656/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765419 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765394.444057:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765394.444063:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765394.444069:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114294656:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765394.444110:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114294784:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765394.444151:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765394.444172:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765394/real 1775765394]  req@ffff9f14852d3b80 x1862022114294784/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765419 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765394.444195:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3b80 x1862022114294784/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765419 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765394.444207:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765394.444213:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765394.444220:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114294784:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765394.444231:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114294912:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765394.444295:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765394.444317:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765394/real 1775765394]  req@ffff9f14852d0000 x1862022114294912/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765419 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765394.444356:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0000 x1862022114294912/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765419 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765394.444373:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765394.444379:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765394.444386:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114294912:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775765395.466641:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:1.0:1775765396.490632:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:1.0:1775765397.514638:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:1.0:1775765398.538612:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:1.0:1775765399.562654:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:1.0:1775765399.562921:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114295040:192.168.202.110@tcp:250:kworker.0
00000100:00100000:1.0:1775765399.563049:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114295168:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765399.563087:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114295296:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765399.563114:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114295424:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775765399.563138:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114295552:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775765399.563166:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114295680:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765399.563191:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114295808:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765399.563253:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114295936:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775765400.586649:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:1.0:1775765401.610714:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:1.0:1775765402.634705:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:1.0:1775765402.635093:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765399/real 1775765402]  req@ffff9f14a04d2300 x1862022114295936/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765424 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765402.635126:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d2300 x1862022114295936/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765424 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765402.635139:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765402.635145:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765402.635150:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114295936:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775765402.635160:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765399/real 1775765402]  req@ffff9f14a04d3800 x1862022114295808/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765424 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765402.635175:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3800 x1862022114295808/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765424 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765402.635192:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765402.635197:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765402.635201:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114295808:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765402.635212:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765399/real 1775765402]  req@ffff9f14a04d1180 x1862022114295552/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765424 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765402.635227:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1180 x1862022114295552/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765424 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765402.635234:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765402.635238:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765402.635242:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114295552:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775765402.635251:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765399/real 1775765402]  req@ffff9f14a04d2d80 x1862022114295680/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765424 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765402.635263:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d2d80 x1862022114295680/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765424 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765402.635272:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765402.635276:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765402.635280:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114295680:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765402.635287:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765399/real 1775765402]  req@ffff9f14a04d0700 x1862022114295168/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765424 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765402.635299:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0700 x1862022114295168/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765424 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765402.635306:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765402.635310:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765402.635313:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114295168:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765402.635321:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765399/real 1775765402]  req@ffff9f14a04d1500 x1862022114295296/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765424 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765402.635334:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1500 x1862022114295296/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765424 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765402.635342:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765402.635345:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765402.635348:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114295296:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765402.635356:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765399/real 1775765402]  req@ffff9f14a04d0000 x1862022114295424/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765424 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765402.635369:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0000 x1862022114295424/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765424 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765402.635377:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765402.635381:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765402.635384:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114295424:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775765402.635392:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765399/real 1775765402]  req@ffff9f14a04d3480 x1862022114295040/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765424 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765402.635407:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3480 x1862022114295040/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765424 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765402.635415:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765402.635419:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765402.635423:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114295040:192.168.202.110@tcp:250:kworker.0
00000100:00100000:1.0:1775765403.658620:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:1.0:1775765404.682643:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765404.682668:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1775765404.682671:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765404.682683:0:73045:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 2782
00000100:00080000:1.0:1775765404.682698:0:73045:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765404.682704:0:73045:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775765404.682736:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fce00 x1862022114296064/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765404.682759:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765404.682762:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775765404.682763:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765404.682766:0:73045:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2782
00000100:00080000:1.0:1775765404.682769:0:73045:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765404.682772:0:73045:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765404.682780:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775765404.682790:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdc00 x1862022114296192/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765404.682800:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765404.682807:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775765404.682810:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765404.682821:0:73045:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2782
00000100:00080000:1.0:1775765404.682823:0:73045:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765404.682826:0:73045:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765404.682831:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775765404.682844:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ffb80 x1862022114296320/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765404.682855:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765404.682859:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775765404.682861:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765404.682865:0:73045:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2782
00000100:00080000:1.0:1775765404.682870:0:73045:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765404.682879:0:73045:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765404.682886:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775765404.682900:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fc700 x1862022114296448/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765404.682911:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765404.682915:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1775765404.682918:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765404.682928:0:73045:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2782
00000100:00080000:1.0:1775765404.682935:0:73045:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765404.682941:0:73045:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765404.682947:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775765404.682965:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdf80 x1862022114296576/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765404.682985:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765404.682990:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775765404.682993:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765404.682999:0:73045:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 2782
00000100:00080000:1.0:1775765404.683004:0:73045:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765404.683012:0:73045:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765404.683023:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775765404.683039:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fd880 x1862022114296704/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765404.683054:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765404.683059:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775765404.683061:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765404.683069:0:73045:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 2782
00000100:00080000:1.0:1775765404.683076:0:73045:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765404.683085:0:73045:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765404.683102:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775765404.683121:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff480 x1862022114296832/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765404.683141:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765404.683145:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775765404.683147:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765404.683157:0:73045:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 2782
00000100:00080000:1.0:1775765404.683163:0:73045:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765404.683171:0:73045:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765404.683182:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775765404.683201:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fce00 x1862022114296960/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765414.922670:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765414.922679:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1775765414.922684:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765414.922697:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 2792
00000100:00080000:1.0:1775765414.922712:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765414.922721:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775765414.922767:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fe300 x1862022114298112/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765414.922794:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765414.922797:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775765414.922798:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765414.922802:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2792
00000100:00080000:1.0:1775765414.922805:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765414.922807:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765414.922816:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775765414.922828:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff100 x1862022114298240/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765414.922837:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765414.922841:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775765414.922844:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765414.922850:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2792
00000100:00080000:1.0:1775765414.922853:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765414.922856:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765414.922859:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775765414.922868:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fea00 x1862022114298368/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765414.922874:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765414.922877:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775765414.922878:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765414.922881:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2792
00000100:00080000:1.0:1775765414.922884:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765414.922886:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765414.922889:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775765414.922896:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fc380 x1862022114298496/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765414.922901:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765414.922903:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1775765414.922904:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765414.922906:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2792
00000100:00080000:1.0:1775765414.922909:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765414.922912:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765414.922916:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775765414.922922:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fca80 x1862022114298624/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765414.922927:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765414.922930:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775765414.922932:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765414.922939:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 2792
00000100:00080000:1.0:1775765414.922944:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765414.922947:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765414.922950:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775765414.922957:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fed80 x1862022114298752/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765414.922963:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765414.922966:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775765414.922967:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765414.922970:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 2792
00000100:00080000:1.0:1775765414.922972:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765414.922974:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765414.922977:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775765414.922983:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff800 x1862022114298880/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765414.922989:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765414.922992:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775765414.922993:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765414.922998:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 2792
00000100:00080000:1.0:1775765414.923000:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765414.923007:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765414.923013:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775765414.923031:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fce00 x1862022114299008/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:1.0:1775765417.994697:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:1.0:1775765419.018634:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:1.0:1775765420.042650:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765420.042660:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1775765420.042664:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765420.042675:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 2797
00000100:00080000:1.0:1775765420.042684:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765420.042690:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775765420.042728:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff480 x1862022114299136/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765420.042752:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765420.042755:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775765420.042757:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765420.042760:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2797
00000100:00080000:1.0:1775765420.042763:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765420.042766:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765420.042774:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775765420.042783:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fd880 x1862022114299264/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765420.042790:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765420.042793:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775765420.042795:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765420.042798:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2797
00000100:00080000:1.0:1775765420.042800:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765420.042802:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765420.042806:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775765420.042815:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdf80 x1862022114299392/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765420.042822:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765420.042824:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775765420.042825:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765420.042829:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2797
00000100:00080000:1.0:1775765420.042831:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765420.042834:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765420.042837:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775765420.042845:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fc700 x1862022114299520/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765420.042851:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765420.042853:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1775765420.042854:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765420.042857:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2797
00000100:00080000:1.0:1775765420.042862:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765420.042864:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765420.042867:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775765420.042876:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ffb80 x1862022114299648/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765420.042884:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765420.042886:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775765420.042888:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765420.042891:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 2797
00000100:00080000:1.0:1775765420.042899:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765420.042902:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765420.042905:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775765420.042912:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdc00 x1862022114299776/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765420.042918:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765420.042921:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775765420.042923:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765420.042937:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 2797
00000100:00080000:1.0:1775765420.042942:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765420.042945:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765420.042949:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775765420.042957:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fce00 x1862022114299904/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765420.042964:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765420.042969:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775765420.042972:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765420.042979:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 2797
00000100:00080000:1.0:1775765420.042984:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765420.042990:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765420.042993:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775765420.043000:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff800 x1862022114300032/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765425.162619:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765425.162627:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1775765425.162631:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765425.162639:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 2802
00000100:00080000:1.0:1775765425.162648:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765425.162655:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775765425.162693:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fed80 x1862022114300160/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765425.162723:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765425.162726:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775765425.162727:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765425.162731:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2802
00000100:00080000:1.0:1775765425.162734:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765425.162737:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765425.162745:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775765425.162756:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fca80 x1862022114300288/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765425.162763:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765425.162767:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775765425.162768:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765425.162771:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2802
00000100:00080000:1.0:1775765425.162773:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765425.162775:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765425.162778:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775765425.162785:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fc380 x1862022114300416/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765425.162794:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765425.162800:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775765425.162802:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765425.162805:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2802
00000100:00080000:1.0:1775765425.162808:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765425.162810:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765425.162813:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775765425.162821:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fea00 x1862022114300544/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765425.162827:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765425.162829:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1775765425.162830:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765425.162833:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2802
00000100:00080000:1.0:1775765425.162836:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765425.162838:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765425.162841:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775765425.162847:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff100 x1862022114300672/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765425.162854:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765425.162856:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775765425.162857:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765425.162862:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 2802
00000100:00080000:1.0:1775765425.162865:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765425.162869:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765425.162872:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775765425.162878:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fe300 x1862022114300800/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765425.162885:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765425.162888:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775765425.162889:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765425.162892:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 2802
00000100:00080000:1.0:1775765425.162894:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765425.162896:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765425.162900:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775765425.162911:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff800 x1862022114300928/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765425.162939:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765425.162942:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775765425.162944:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765425.162948:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 2802
00000100:00080000:1.0:1775765425.162952:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765425.162960:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765425.162967:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775765425.162977:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fce00 x1862022114301056/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:1.0:1775765427.210652:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:1.0:1775765428.234702:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:1.0:1775765429.258667:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:1.0:1775765430.282676:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765430.282689:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1775765430.282696:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765430.282709:0:73045:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 2807
00000100:00080000:1.0:1775765430.282724:0:73045:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765430.282736:0:73045:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775765430.282788:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdc00 x1862022114301184/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765430.282843:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765430.282848:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775765430.282850:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765430.282855:0:73045:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2807
00000100:00080000:1.0:1775765430.282859:0:73045:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765430.282863:0:73045:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765430.282876:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775765430.282894:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ffb80 x1862022114301312/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765430.282907:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765430.282912:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775765430.282914:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765430.282920:0:73045:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2807
00000100:00080000:1.0:1775765430.282923:0:73045:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765430.282927:0:73045:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765430.282933:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775765430.282948:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fc700 x1862022114301440/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765430.282976:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765430.282980:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775765430.282982:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765430.282987:0:73045:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2807
00000100:00080000:1.0:1775765430.282991:0:73045:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765430.282995:0:73045:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765430.283001:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775765430.283017:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdf80 x1862022114301568/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765430.283028:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765430.283032:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1775765430.283034:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765430.283038:0:73045:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2807
00000100:00080000:1.0:1775765430.283055:0:73045:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765430.283060:0:73045:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765430.283066:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775765430.283088:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fd880 x1862022114301696/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765430.283101:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765430.283106:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775765430.283109:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765430.283114:0:73045:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 2807
00000100:00080000:1.0:1775765430.283118:0:73045:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765430.283123:0:73045:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765430.283130:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775765430.283149:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff480 x1862022114301824/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765430.283213:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765430.283220:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775765430.283225:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765430.283239:0:73045:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 2807
00000100:00080000:1.0:1775765430.283252:0:73045:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765430.283268:0:73045:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765430.283283:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775765430.283317:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fce00 x1862022114301952/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765430.283346:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765430.283350:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775765430.283353:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765430.283360:0:73045:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 2807
00000100:00080000:1.0:1775765430.283369:0:73045:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765430.283374:0:73045:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765430.283384:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775765430.283435:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff800 x1862022114302080/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:1.0:1775765433.354765:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:1.0:1775765434.378681:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:1.0:1775765435.402643:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:1.0:1775765436.426655:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:1.0:1775765437.450723:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:1.0:1775765438.474726:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:1.0:1775765439.498666:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775765440.523109:0:73045:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 2818
00000100:00080000:3.0:1775765440.523113:0:73045:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765440.523116:0:73045:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765440.523121:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775765440.523195:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746680 x1862022114304000/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765440.523206:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765440.523209:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775765440.523211:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765440.523215:0:73045:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 2818
00000100:00080000:3.0:1775765440.523220:0:73045:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765440.523224:0:73045:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765440.523228:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775765440.523239:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745880 x1862022114304128/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765445.642701:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765445.642708:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1775765445.642713:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765445.642724:0:73045:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 2823
00000100:00080000:1.0:1775765445.642736:0:73045:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765445.642743:0:73045:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775765445.642790:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fe300 x1862022114304256/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765445.642820:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765445.642841:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775765445.642842:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765445.642857:0:73045:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2823
00000100:00080000:1.0:1775765445.642861:0:73045:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765445.642865:0:73045:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765445.642876:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775765445.642893:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff100 x1862022114304384/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765445.642902:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765445.642905:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775765445.642907:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765445.642910:0:73045:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2823
00000100:00080000:1.0:1775765445.642913:0:73045:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765445.642915:0:73045:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765445.642919:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775765445.642928:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fea00 x1862022114304512/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765445.642934:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765445.642937:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775765445.642938:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765445.642941:0:73045:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2823
00000100:00080000:1.0:1775765445.642952:0:73045:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765445.642955:0:73045:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765445.642960:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775765445.642972:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fc380 x1862022114304640/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765445.642980:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765445.642983:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1775765445.642985:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765445.642988:0:73045:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2823
00000100:00080000:1.0:1775765445.642990:0:73045:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765445.642993:0:73045:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765445.642996:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775765445.643003:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fca80 x1862022114304768/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765445.643010:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765445.643013:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775765445.643016:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765445.643024:0:73045:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 2823
00000100:00080000:1.0:1775765445.643031:0:73045:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765445.643037:0:73045:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765445.643041:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775765445.643049:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fed80 x1862022114304896/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765445.643055:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765445.643058:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775765445.643061:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765445.643065:0:73045:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 2823
00000100:00080000:1.0:1775765445.643069:0:73045:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765445.643075:0:73045:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765445.643081:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775765445.643094:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff800 x1862022114305024/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765445.643105:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765445.643110:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775765445.643111:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765445.643118:0:73045:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 2823
00000100:00080000:1.0:1775765445.643124:0:73045:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765445.643127:0:73045:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765445.643131:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775765445.643138:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fce00 x1862022114305152/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765450.762829:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765450.762842:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775765450.762849:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765450.762864:0:73045:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 2828
00000100:00080000:3.0:1775765450.762881:0:73045:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765450.762891:0:73045:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775765450.762967:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744a80 x1862022114305280/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765450.763008:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765450.763017:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775765450.763020:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765450.763026:0:73045:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2828
00000100:00080000:3.0:1775765450.763030:0:73045:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765450.763035:0:73045:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765450.763060:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775765450.763079:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745880 x1862022114305408/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765450.763092:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765450.763097:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775765450.763100:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765450.763105:0:73045:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2828
00000100:00080000:3.0:1775765450.763110:0:73045:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765450.763120:0:73045:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765450.763128:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775765450.763150:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746680 x1862022114305536/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765450.763164:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765450.763168:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775765450.763171:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765450.763176:0:73045:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2828
00000100:00080000:3.0:1775765450.763180:0:73045:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765450.763185:0:73045:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765450.763196:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775765450.763261:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744e00 x1862022114305664/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765450.763284:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765450.763291:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775765450.763294:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765450.763302:0:73045:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2828
00000100:00080000:3.0:1775765450.763311:0:73045:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765450.763318:0:73045:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765450.763327:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775765450.763345:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747b80 x1862022114305792/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765450.763364:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765450.763383:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775765450.763391:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765450.763401:0:73045:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 2828
00000100:00080000:3.0:1775765450.763408:0:73045:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765450.763422:0:73045:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765450.763438:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775765450.763457:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746300 x1862022114305920/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765450.763471:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765450.763476:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775765450.763480:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765450.763496:0:73045:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 2828
00000100:00080000:3.0:1775765450.763502:0:73045:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765450.763511:0:73045:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765450.763527:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775765450.763543:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746a00 x1862022114306048/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765450.763565:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765450.763617:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775765450.763621:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765450.763628:0:73045:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 2828
00000100:00080000:3.0:1775765450.763636:0:73045:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765450.763644:0:73045:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765450.763655:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775765450.763677:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744380 x1862022114306176/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775765461.002752:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:1.0:1775765461.002981:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114307328:192.168.202.110@tcp:250:kworker.0
00000800:00000100:1.0:1775765461.003121:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765461.003183:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765461/real 1775765461]  req@ffff9f14852d0380 x1862022114307328/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765486 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765461.003207:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0380 x1862022114307328/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765486 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765461.003252:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765461.003273:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765461.003286:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114307328:192.168.202.110@tcp:250:kworker.0
00000100:00100000:1.0:1775765461.003335:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114307456:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765461.003371:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765461.003389:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765461/real 1775765461]  req@ffff9f14852d3800 x1862022114307456/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765486 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765461.003419:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3800 x1862022114307456/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765486 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765461.003428:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765461.003442:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765461.003461:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114307456:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765461.003499:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114307584:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765461.003587:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765461.003608:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765461/real 1775765461]  req@ffff9f14852d2d80 x1862022114307584/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765486 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765461.003638:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2d80 x1862022114307584/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765486 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765461.003660:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765461.003667:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765461.003670:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114307584:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765461.003681:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114307712:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765461.003714:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765461.003729:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765461/real 1775765461]  req@ffff9f14852d0700 x1862022114307712/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765486 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765461.003739:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0700 x1862022114307712/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765486 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765461.003748:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765461.003752:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765461.003755:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114307712:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775765461.003772:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114307840:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765461.003783:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765461.003790:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765461/real 1775765461]  req@ffff9f14852d0a80 x1862022114307840/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765486 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765461.003798:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0a80 x1862022114307840/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765486 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765461.003803:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765461.003806:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765461.003808:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114307840:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775765461.003812:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114307968:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765461.003822:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765461.003828:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765461/real 1775765461]  req@ffff9f14852d1500 x1862022114307968/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765486 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765461.003840:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1500 x1862022114307968/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765486 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765461.003845:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765461.003847:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765461.003849:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114307968:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765461.003853:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114308096:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765461.003875:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765461.003880:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765461/real 1775765461]  req@ffff9f14852d2680 x1862022114308096/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765486 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765461.003888:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2680 x1862022114308096/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765486 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765461.003896:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765461.003900:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765461.003903:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114308096:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765461.003910:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114308224:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765461.003933:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765461.003944:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765461/real 1775765461]  req@ffff9f14852d2a00 x1862022114308224/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765486 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765461.003957:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2a00 x1862022114308224/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765486 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765461.003967:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765461.003970:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765461.003972:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114308224:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775765462.026656:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:3.0:1775765463.050611:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:3.0:1775765464.074657:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:3.0:1775765465.098649:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:3.0:1775765466.122622:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:1.0:1775765466.122709:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765466.122719:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1775765466.122724:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765466.122736:0:73045:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 2843
00000100:00080000:1.0:1775765466.122750:0:73045:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765466.122757:0:73045:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775765466.122807:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff480 x1862022114308352/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765466.122842:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765466.122846:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775765466.122847:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765466.122851:0:73045:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2843
00000100:00080000:1.0:1775765466.122855:0:73045:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765466.122858:0:73045:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765466.122870:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775765466.122882:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fd880 x1862022114308480/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765466.122895:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765466.122907:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775765466.122910:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765466.122915:0:73045:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2843
00000100:00080000:1.0:1775765466.122917:0:73045:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765466.122922:0:73045:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765466.122926:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775765466.122936:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdf80 x1862022114308608/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765466.122943:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765466.122946:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775765466.122948:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765466.122952:0:73045:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2843
00000100:00080000:1.0:1775765466.122955:0:73045:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765466.122958:0:73045:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765466.122964:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775765466.122974:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fc700 x1862022114308736/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765466.122981:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765466.122985:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1775765466.122987:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765466.122991:0:73045:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2843
00000100:00080000:1.0:1775765466.122996:0:73045:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765466.122999:0:73045:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765466.123004:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775765466.123020:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ffb80 x1862022114308864/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765466.123049:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765466.123052:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775765466.123053:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765466.123057:0:73045:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 2843
00000100:00080000:1.0:1775765466.123078:0:73045:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765466.123083:0:73045:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765466.123088:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775765466.123119:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdc00 x1862022114308992/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765466.123128:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765466.123151:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775765466.123153:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765466.123161:0:73045:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 2843
00000100:00080000:1.0:1775765466.123173:0:73045:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765466.123178:0:73045:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765466.123186:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775765466.123200:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fce00 x1862022114309120/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765466.123209:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765466.123213:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775765466.123216:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765466.123223:0:73045:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 2843
00000100:00080000:1.0:1775765466.123230:0:73045:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765466.123242:0:73045:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765466.123252:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775765466.123280:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff800 x1862022114309248/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775765467.146628:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:3.0:1775765468.170683:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:3.0:1775765469.194703:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:1.0:1775765469.195286:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765466/real 1775765469]  req@ffff9f14ab4ff800 x1862022114309248/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765491 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765469.195320:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff800 x1862022114309248/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765491 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765469.195335:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765469.195341:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765469.195347:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114309248:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775765469.195358:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765466/real 1775765469]  req@ffff9f14ab4ffb80 x1862022114308864/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765491 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765469.195373:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ffb80 x1862022114308864/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765491 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765469.195381:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765469.195386:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765469.195390:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ffb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114308864:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775765469.195402:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765466/real 1775765469]  req@ffff9f14ab4fdc00 x1862022114308992/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765491 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765469.195439:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdc00 x1862022114308992/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765491 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765469.195455:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765469.195462:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765469.195468:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114308992:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765469.195494:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765466/real 1775765469]  req@ffff9f14ab4fce00 x1862022114309120/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765491 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765469.195519:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fce00 x1862022114309120/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765491 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765469.195530:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765469.195536:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765469.195542:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114309120:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765469.195551:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765466/real 1775765469]  req@ffff9f14ab4fd880 x1862022114308480/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765491 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765469.195617:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fd880 x1862022114308480/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765491 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765469.195631:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765469.195639:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765469.195647:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114308480:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765469.195659:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765466/real 1775765469]  req@ffff9f14ab4fdf80 x1862022114308608/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765491 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765469.195679:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdf80 x1862022114308608/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765491 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765469.195691:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765469.195697:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765469.195703:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114308608:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765469.195716:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765466/real 1775765469]  req@ffff9f14ab4fc700 x1862022114308736/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765491 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765469.195741:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fc700 x1862022114308736/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765491 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765469.195758:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765469.195766:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765469.195773:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fc700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114308736:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775765469.195784:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765466/real 1775765469]  req@ffff9f14ab4ff480 x1862022114308352/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765491 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765469.195813:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff480 x1862022114308352/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765491 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765469.195824:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765469.195830:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765469.195834:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114308352:192.168.202.110@tcp:250:kworker.0
00000100:00100000:3.0:1775765470.218661:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:3.0:1775765471.242662:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:1.0:1775765471.242928:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114309376:192.168.202.110@tcp:250:kworker.0
00000800:00000100:1.0:1775765471.243038:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765471.243117:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765471/real 1775765471]  req@ffff9f14852d1f80 x1862022114309376/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765496 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765471.243182:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1f80 x1862022114309376/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765496 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765471.243211:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765471.243223:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765471.243232:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114309376:192.168.202.110@tcp:250:kworker.0
00000100:00100000:1.0:1775765471.243267:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114309504:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765471.243315:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765471.243339:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765471/real 1775765471]  req@ffff9f14852d1180 x1862022114309504/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765496 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765471.243375:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1180 x1862022114309504/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765496 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765471.243398:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765471.243408:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765471.243414:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114309504:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765471.243427:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114309632:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765471.243472:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765471.243489:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765471/real 1775765471]  req@ffff9f14852d2a00 x1862022114309632/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765496 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765471.243514:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2a00 x1862022114309632/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765496 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765471.243526:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765471.243529:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765471.243533:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114309632:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765471.243541:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114309760:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765471.243569:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765471.243612:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765471/real 1775765471]  req@ffff9f14852d2680 x1862022114309760/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765496 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765471.243631:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2680 x1862022114309760/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765496 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765471.243643:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765471.243647:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765471.243651:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114309760:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775765471.243659:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114309888:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765471.243687:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765471.243701:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765471/real 1775765471]  req@ffff9f14852d1500 x1862022114309888/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765496 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765471.243716:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1500 x1862022114309888/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765496 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765471.243734:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765471.243739:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765471.243743:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114309888:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775765471.243766:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114310016:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765471.243803:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765471.243817:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765471/real 1775765471]  req@ffff9f14852d0a80 x1862022114310016/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765496 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765471.243840:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0a80 x1862022114310016/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765496 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765471.243850:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765471.243854:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765471.243858:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114310016:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765471.243866:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114310144:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765471.243893:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765471.243934:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765471/real 1775765471]  req@ffff9f14852d0700 x1862022114310144/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765496 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765471.243951:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0700 x1862022114310144/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765496 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765471.243962:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765471.243967:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765471.243972:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114310144:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765471.243981:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114310272:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765471.244037:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765471.244053:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765471/real 1775765471]  req@ffff9f14852d2d80 x1862022114310272/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765496 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765471.244069:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2d80 x1862022114310272/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765496 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765471.244080:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765471.244084:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765471.244088:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114310272:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775765472.266704:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:3.0:1775765473.290648:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:3.0:1775765474.314670:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:3.0:1775765475.338682:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775765475.594874:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775765476.362754:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:1.0:1775765476.362998:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114310400:192.168.202.110@tcp:250:kworker.0
00000800:00000100:1.0:1775765476.363110:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765476.363193:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765476/real 1775765476]  req@ffff9f14852d3800 x1862022114310400/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765501 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765476.363230:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3800 x1862022114310400/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765501 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765476.363249:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765476.363258:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765476.363275:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114310400:192.168.202.110@tcp:250:kworker.0
00000100:00100000:1.0:1775765476.363318:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114310528:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765476.363366:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765476.363382:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765476/real 1775765476]  req@ffff9f14852d0380 x1862022114310528/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765501 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765476.363440:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0380 x1862022114310528/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765501 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765476.363457:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765476.363464:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765476.363469:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114310528:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765476.363483:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114310656:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765476.363525:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765476.363545:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765476/real 1775765476]  req@ffff9f14852d0000 x1862022114310656/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765501 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765476.363601:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0000 x1862022114310656/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765501 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765476.363622:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765476.363633:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765476.363641:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114310656:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765476.363651:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114310784:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765476.363694:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765476.363708:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765476/real 1775765476]  req@ffff9f14852d3b80 x1862022114310784/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765501 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765476.363760:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3b80 x1862022114310784/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765501 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765476.363832:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765476.363838:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765476.363855:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114310784:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775765476.363892:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114310912:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765476.363950:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765476.363984:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765476/real 1775765476]  req@ffff9f14852d3100 x1862022114310912/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765501 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765476.363998:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3100 x1862022114310912/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765501 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765476.364007:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765476.364014:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765476.364019:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114310912:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775765476.364029:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114311040:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765476.364054:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765476.364065:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765476/real 1775765476]  req@ffff9f14852d2300 x1862022114311040/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765501 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765476.364074:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2300 x1862022114311040/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765501 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765476.364084:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765476.364087:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765476.364089:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114311040:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765476.364094:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114311168:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765476.364107:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765476.364114:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765476/real 1775765476]  req@ffff9f14852d3480 x1862022114311168/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765501 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765476.364121:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3480 x1862022114311168/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765501 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765476.364126:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765476.364128:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765476.364130:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114311168:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765476.364142:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114311296:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765476.364153:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765476.364160:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765476/real 1775765476]  req@ffff9f14852d1c00 x1862022114311296/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765501 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765476.364168:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1c00 x1862022114311296/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765501 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765476.364177:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765476.364180:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765476.364184:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114311296:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775765477.386698:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:3.0:1775765478.410659:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775765479.690828:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765480.458699:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:2.0:1775765481.482721:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765481.482733:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1775765481.482739:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765481.482751:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 2858
00000100:00080000:2.0:1775765481.482765:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765481.482775:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775765481.482830:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:2.0:1775765481.482841:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0e00 x1862022114311424/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775765481.482879:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765481.482884:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775765481.482887:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765481.482893:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2858
00000100:00080000:2.0:1775765481.482897:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765481.482904:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775765481.482937:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775765481.482970:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1880 x1862022114311552/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:1.0:1775765481.482987:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114311424:192.168.202.110@tcp:250:kworker.0
00000100:00080000:2.0:1775765481.482992:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765481.483010:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775765481.483012:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765481.483019:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2858
00000100:00080000:2.0:1775765481.483025:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765481.483031:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775765481.483039:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775765481.483057:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1c00 x1862022114311680/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775765481.483073:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765481.483077:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775765481.483079:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765481.483083:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2858
00000100:00080000:2.0:1775765481.483087:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765481.483091:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775765481.483097:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:1.0:1775765481.483112:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775765481.483112:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3480 x1862022114311808/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775765481.483130:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765481.483135:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1775765481.483138:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765481.483143:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2858
00000100:00080000:2.0:1775765481.483148:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765481.483153:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775765481.483159:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775765481.483174:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2300 x1862022114311936/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:1.0:1775765481.483184:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765481/real 1775765481]  req@ffff9f14852d0e00 x1862022114311424/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765506 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765481.483207:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765481.483230:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775765481.483236:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765481.483244:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 2858
00000100:00080000:2.0:1775765481.483252:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:1.0:1775765481.483262:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0e00 x1862022114311424/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765506 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765481.483263:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775765481.483281:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00080000:1.0:1775765481.483303:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00100000:2.0:1775765481.483314:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3100 x1862022114312064/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765481.483323:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765481.483345:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114311424:192.168.202.110@tcp:250:kworker.0
00000100:00080000:2.0:1775765481.483359:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765481.483368:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775765481.483373:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765481.483382:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 2858
00000100:00080000:2.0:1775765481.483390:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:1.0:1775765481.483396:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114311552:192.168.202.110@tcp:38:kworker.0
00000100:00080000:2.0:1775765481.483405:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775765481.483417:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000800:00000100:1.0:1775765481.483437:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775765481.483450:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3b80 x1862022114312192/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:1.0:1775765481.483461:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765481/real 1775765481]  req@ffff9f14852d1880 x1862022114311552/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765506 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765481.483531:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1880 x1862022114311552/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765506 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765481.483531:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765481.483561:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765481.483572:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765481.483578:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114311552:192.168.202.110@tcp:38:kworker.0
00000100:00080000:2.0:1775765481.483594:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775765481.483599:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765481.483608:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 2858
00000100:00080000:2.0:1775765481.483614:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765481.483619:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775765481.483629:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775765481.483643:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114311680:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775765481.483658:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0000 x1862022114312320/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:1.0:1775765481.483702:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765481.483734:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765481/real 1775765481]  req@ffff9f14852d1c00 x1862022114311680/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765506 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765481.483751:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1c00 x1862022114311680/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765506 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765481.483762:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765481.483767:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765481.483771:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114311680:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765481.483779:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114311808:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765481.483803:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765481.483814:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765481/real 1775765481]  req@ffff9f14852d3480 x1862022114311808/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765506 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765481.483828:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3480 x1862022114311808/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765506 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765481.483838:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765481.483841:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765481.483845:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114311808:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775765481.483851:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114311936:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765481.483870:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765481.483881:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765481/real 1775765481]  req@ffff9f14852d2300 x1862022114311936/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765506 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765481.483931:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2300 x1862022114311936/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765506 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765481.483938:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765481.483941:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765481.483943:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114311936:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775765481.483948:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114312064:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765481.483965:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765481.483973:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765481/real 1775765481]  req@ffff9f14852d3100 x1862022114312064/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765506 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765481.483982:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3100 x1862022114312064/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765506 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765481.483988:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765481.483991:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765481.483993:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114312064:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765481.484014:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114312192:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765481.484026:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765481.484033:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765481/real 1775765481]  req@ffff9f14852d3b80 x1862022114312192/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765506 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765481.484041:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3b80 x1862022114312192/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765506 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765481.484053:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765481.484055:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765481.484058:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114312192:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765481.484062:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114312320:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765481.484074:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765481.484080:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765481/real 1775765481]  req@ffff9f14852d0000 x1862022114312320/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765506 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765481.484088:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0000 x1862022114312320/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765506 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765481.484094:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765481.484096:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765481.484098:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114312320:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775765482.506665:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775765482.762776:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775765483.530718:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:3.0:1775765484.554668:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:3.0:1775765485.578664:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:2.0:1775765486.602747:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:3.0:1775765486.602751:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:2.0:1775765486.602760:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1775765486.602765:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765486.602774:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 2864
00000100:00080000:2.0:1775765486.602786:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765486.602794:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775765486.602833:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0380 x1862022114312448/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775765486.602902:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765486.602906:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775765486.602908:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765486.602923:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2864
00000100:00080000:2.0:1775765486.602927:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765486.602930:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775765486.602939:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775765486.602950:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3800 x1862022114312576/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775765486.602959:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765486.602963:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775765486.602964:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765486.602968:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2864
00000100:00080000:2.0:1775765486.602971:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765486.602974:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775765486.602980:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775765486.602988:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114312448:192.168.202.110@tcp:250:kworker.0
00000100:00100000:2.0:1775765486.602990:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2d80 x1862022114312704/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775765486.603006:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765486.603009:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775765486.603010:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765486.603013:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2864
00000100:00080000:2.0:1775765486.603017:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765486.603020:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775765486.603023:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775765486.603033:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0700 x1862022114312832/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775765486.603042:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765486.603045:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1775765486.603046:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765486.603049:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2864
00000100:00080000:2.0:1775765486.603052:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765486.603055:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775765486.603059:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:1.0:1775765486.603069:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775765486.603070:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0a80 x1862022114312960/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775765486.603081:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765486.603084:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775765486.603086:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765486.603091:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 2864
00000100:00080000:2.0:1775765486.603094:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765486.603097:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775765486.603101:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775765486.603111:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1500 x1862022114313088/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:1.0:1775765486.603120:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765486/real 1775765486]  req@ffff9f14852d0380 x1862022114312448/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765511 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765486.603143:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765486.603147:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775765486.603149:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765486.603154:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 2864
00000100:00080000:2.0:1775765486.603157:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:1.0:1775765486.603158:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0380 x1862022114312448/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765511 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765486.603166:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775765486.603174:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00080000:1.0:1775765486.603179:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00100000:2.0:1775765486.603187:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2680 x1862022114313216/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765486.603188:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765486.603210:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114312448:192.168.202.110@tcp:250:kworker.0
00000100:00080000:2.0:1775765486.603211:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765486.603220:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775765486.603223:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:1.0:1775765486.603228:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114312576:192.168.202.110@tcp:38:kworker.0
00000100:00080000:2.0:1775765486.603229:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 2864
00000100:00080000:2.0:1775765486.603234:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765486.603239:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775765486.603243:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775765486.603257:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2a00 x1862022114313344/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:1.0:1775765486.603267:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765486.603285:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765486/real 1775765486]  req@ffff9f14852d3800 x1862022114312576/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765511 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765486.603299:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3800 x1862022114312576/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765511 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765486.603307:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765486.603310:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765486.603312:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114312576:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765486.603334:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114312704:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765486.603348:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765486.603355:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765486/real 1775765486]  req@ffff9f14852d2d80 x1862022114312704/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765511 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765486.603370:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2d80 x1862022114312704/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765511 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765486.603377:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765486.603379:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765486.603382:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114312704:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765486.603387:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114312832:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765486.603400:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765486.603425:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765486/real 1775765486]  req@ffff9f14852d0700 x1862022114312832/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765511 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765486.603439:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0700 x1862022114312832/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765511 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765486.603458:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765486.603461:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765486.603464:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114312832:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775765486.603471:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114312960:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765486.603491:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765486.603502:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765486/real 1775765486]  req@ffff9f14852d0a80 x1862022114312960/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765511 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765486.603516:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0a80 x1862022114312960/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765511 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765486.603525:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765486.603528:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765486.603531:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114312960:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775765486.603538:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114313088:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765486.603555:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765486.603622:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765486/real 1775765486]  req@ffff9f14852d1500 x1862022114313088/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765511 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765486.603639:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1500 x1862022114313088/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765511 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765486.603650:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765486.603654:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765486.603658:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114313088:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765486.603665:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114313216:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765486.603704:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765486.603713:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765486/real 1775765486]  req@ffff9f14852d2680 x1862022114313216/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765511 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765486.603722:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2680 x1862022114313216/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765511 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765486.603729:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765486.603731:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765486.603733:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114313216:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765486.603738:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114313344:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765486.603765:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765486.603774:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765486/real 1775765486]  req@ffff9f14852d2a00 x1862022114313344/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765511 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765486.603784:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2a00 x1862022114313344/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765511 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765486.603790:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765486.603793:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765486.603796:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114313344:192.168.202.110@tcp:8:kworker.0
00010000:00010000:1.0:1775765486.858809:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775765487.626667:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:3.0:1775765488.650689:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:2.0:1775765489.674766:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775765489.930784:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765490.698650:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:2.0:1775765491.722674:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765491.722683:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1775765491.722688:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765491.722698:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 2869
00000100:00080000:2.0:1775765491.722710:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765491.722718:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775765491.722736:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:2.0:1775765491.722773:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1180 x1862022114313472/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775765491.722803:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765491.722808:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775765491.722810:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:1.0:1775765491.722853:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114313472:192.168.202.110@tcp:250:kworker.0
00000100:00080000:2.0:1775765491.722884:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2869
00000100:00080000:2.0:1775765491.722890:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765491.722895:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775765491.722904:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775765491.722917:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1f80 x1862022114313600/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775765491.722935:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000800:00000100:1.0:1775765491.722942:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:2.0:1775765491.722942:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775765491.722945:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765491.722950:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2869
00000100:00080000:2.0:1775765491.722954:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765491.722957:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775765491.722961:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775765491.722970:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2a00 x1862022114313728/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775765491.722977:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765491.722979:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775765491.722981:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765491.722984:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2869
00000100:00080000:2.0:1775765491.722987:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765491.722993:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775765491.722995:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765491/real 1775765491]  req@ffff9f14852d1180 x1862022114313472/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765516 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:2.0:1775765491.722997:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775765491.723009:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2680 x1862022114313856/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775765491.723021:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765491.723024:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1775765491.723025:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765491.723029:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2869
00000100:00100000:1.0:1775765491.723033:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1180 x1862022114313472/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765516 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765491.723034:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765491.723043:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775765491.723050:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00080000:1.0:1775765491.723054:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00100000:2.0:1775765491.723059:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1500 x1862022114313984/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765491.723060:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765491.723069:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114313472:192.168.202.110@tcp:250:kworker.0
00000100:00080000:2.0:1775765491.723098:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765491.723101:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775765491.723103:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765491.723107:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 2869
00000100:00080000:2.0:1775765491.723110:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765491.723115:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775765491.723120:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114313600:192.168.202.110@tcp:38:kworker.0
00000008:00100000:2.0:1775765491.723121:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775765491.723136:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0a80 x1862022114314112/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775765491.723146:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765491.723150:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775765491.723152:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000800:00000100:1.0:1775765491.723157:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:2.0:1775765491.723159:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 2869
00000100:00080000:2.0:1775765491.723167:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765491.723171:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775765491.723177:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775765491.723182:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765491/real 1775765491]  req@ffff9f14852d1f80 x1862022114313600/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765516 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775765491.723190:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0700 x1862022114314240/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775765491.723205:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:1.0:1775765491.723206:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1f80 x1862022114313600/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765516 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765491.723217:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775765491.723222:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765491.723232:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775765491.723233:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 2869
00000100:00080000:2.0:1775765491.723242:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765491.723245:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00080000:2.0:1775765491.723250:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775765491.723255:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114313600:192.168.202.110@tcp:38:kworker.0
00000008:00100000:2.0:1775765491.723258:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775765491.723272:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2d80 x1862022114314368/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:1.0:1775765491.723279:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114313728:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765491.723310:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765491.723320:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765491/real 1775765491]  req@ffff9f14852d2a00 x1862022114313728/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765516 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765491.723331:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2a00 x1862022114313728/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765516 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765491.723337:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765491.723340:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765491.723343:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114313728:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765491.723348:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114313856:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765491.723362:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765491.723368:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765491/real 1775765491]  req@ffff9f14852d2680 x1862022114313856/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765516 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765491.723377:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2680 x1862022114313856/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765516 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765491.723393:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765491.723396:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765491.723398:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114313856:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775765491.723402:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114313984:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765491.723414:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765491.723421:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765491/real 1775765491]  req@ffff9f14852d1500 x1862022114313984/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765516 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765491.723436:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1500 x1862022114313984/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765516 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765491.723443:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765491.723445:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765491.723447:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114313984:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775765491.723462:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114314112:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765491.723474:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765491.723491:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765491/real 1775765491]  req@ffff9f14852d0a80 x1862022114314112/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765516 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765491.723501:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0a80 x1862022114314112/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765516 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765491.723507:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765491.723509:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765491.723511:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114314112:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765491.723516:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114314240:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765491.723527:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765491.723534:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765491/real 1775765491]  req@ffff9f14852d0700 x1862022114314240/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765516 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765491.723543:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0700 x1862022114314240/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765516 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765491.723581:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765491.723584:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765491.723587:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114314240:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765491.723592:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114314368:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765491.723609:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765491.723617:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765491/real 1775765491]  req@ffff9f14852d2d80 x1862022114314368/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765516 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765491.723627:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2d80 x1862022114314368/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765516 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765491.723633:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765491.723636:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765491.723638:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114314368:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775765492.746625:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:3.0:1775765493.770640:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775765494.026862:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775765494.794681:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:3.0:1775765495.818692:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:3.0:1775765496.842700:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:1.0:1775765496.842885:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114314496:192.168.202.110@tcp:250:kworker.0
00000800:00000100:1.0:1775765496.842943:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765496.842981:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765496/real 1775765496]  req@ffff9f14a04d1f80 x1862022114314496/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765521 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765496.843002:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1f80 x1862022114314496/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765521 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765496.843019:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765496.843026:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765496.843032:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114314496:192.168.202.110@tcp:250:kworker.0
00000100:00100000:1.0:1775765496.843054:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114314624:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765496.843078:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765496.843091:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765496/real 1775765496]  req@ffff9f14a04d0380 x1862022114314624/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765521 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765496.843113:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0380 x1862022114314624/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765521 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765496.843128:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765496.843132:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765496.843138:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114314624:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765496.843149:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114314752:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765496.843164:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765496.843172:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765496/real 1775765496]  req@ffff9f14a04d0a80 x1862022114314752/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765521 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765496.843180:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0a80 x1862022114314752/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765521 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765496.843185:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765496.843187:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765496.843189:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114314752:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765496.843192:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114314880:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765496.843202:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765496.843207:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765496/real 1775765496]  req@ffff9f14a04d2a00 x1862022114314880/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765521 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765496.843214:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d2a00 x1862022114314880/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765521 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765496.843222:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765496.843225:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765496.843228:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114314880:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775765496.843233:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114315008:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765496.843251:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765496.843260:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765496/real 1775765496]  req@ffff9f14a04d0e00 x1862022114315008/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765521 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765496.843272:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0e00 x1862022114315008/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765521 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765496.843279:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765496.843282:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765496.843285:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114315008:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775765496.843290:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114315136:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765496.843308:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765496.843317:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765496/real 1775765496]  req@ffff9f14a04d3480 x1862022114315136/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765521 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765496.843327:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3480 x1862022114315136/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765521 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765496.843334:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765496.843337:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765496.843340:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114315136:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765496.843357:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114315264:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765496.843377:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765496.843387:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765496/real 1775765496]  req@ffff9f14a04d0000 x1862022114315264/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765521 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765496.843395:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0000 x1862022114315264/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765521 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765496.843400:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765496.843402:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765496.843404:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114315264:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765496.843409:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114315392:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765496.843421:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765496.843427:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765496/real 1775765496]  req@ffff9f14a04d1500 x1862022114315392/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765521 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765496.843434:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1500 x1862022114315392/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765521 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765496.843439:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765496.843441:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765496.843443:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114315392:192.168.202.110@tcp:8:kworker.0
00010000:00010000:1.0:1775765497.098829:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775765497.866692:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775765498.122851:0:60133:0:(ldlm_lockd.c:2794:ldlm_bl_get_work()) Got 2 locks of 0 total in blp.  (0 blwis in pool)
00010000:00010000:1.0:1775765498.122872:0:60133:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f14a04c5600/0x23cab7dad48e6342 lrc: 2/0,0 mode: CR/CR res: [0x200000405:0x7e:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x929304787233679 expref: -99 pid: 73979 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0
00000080:00200000:1.0:1775765498.122986:0:60133:0:(vvp_object.c:117:vvp_conf_set()) [0x200000405:0x7e:0x0]: losing layout lock
00010000:00010000:2.0:1775765498.122998:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00010000:00010000:1.0:1775765498.123056:0:60133:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f14ab59a000/0x23cab7dad48e6311 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x929304787233602 expref: -99 pid: 73979 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0
00010000:00010000:1.0:1775765498.123150:0:60133:0:(ldlm_request.c:1438:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f14a04c5600/0x23cab7dad48e6342 lrc: 1/0,0 mode: --/CR res: [0x200000405:0x7e:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x929304787233679 expref: -99 pid: 73979 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0
00010000:00010000:1.0:1775765498.123164:0:60133:0:(ldlm_request.c:1438:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f14ab59a000/0x23cab7dad48e6311 lrc: 1/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x929304787233602 expref: -99 pid: 73979 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0
00010000:00010000:1.0:1775765498.123171:0:60133:0:(ldlm_request.c:1490:ldlm_cancel_pack()) 2 locks packed
00010000:00010000:1.0:1775765498.123210:0:60133:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f14a04c5600/0x23cab7dad48e6342 lrc: 0/0,0 mode: --/CR res: [0x200000405:0x7e:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x929304787233679 expref: -99 pid: 73979 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0
00010000:00010000:1.0:1775765498.123241:0:60133:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9f14902b3800 lock: ffff9f14ab59a000/0x23cab7dad48e6311 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x929304787233602 expref: -99 pid: 73979 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0
00010000:00010000:1.0:1775765498.123289:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00010000:00010000:1.0:1775765498.123296:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00010000:00010000:1.0:1775765498.123305:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1775765498.123338:0:2403:0:(client.c:1830:ptlrpc_send_new_req()) @@@ req waiting for recovery: (FULL != DISCONN)  req@ffff9f14ab4ff480 x1862022114315520/t0(0) o103->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:17/18 lens 328/224 e 0 to 0 dl 0 ref 1 fl Rpc:WQU/200/ffffffff rc 0/-1 job:'ldlm_bl.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765498.123471:0:2404:0:(client.c:1830:ptlrpc_send_new_req()) @@@ req waiting for recovery: (FULL != DISCONN)  req@ffff9f14a04d0700 x1862022114315648/t0(0) o103->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:17/18 lens 328/224 e 0 to 0 dl 0 ref 1 fl Rpc:WQU/200/ffffffff rc 0/-1 job:'ldlm_bl.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775765498.890672:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775765499.146847:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765499.914784:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775765500.170863:0:46896:0:(ldlm_lockd.c:2794:ldlm_bl_get_work()) Got 2 locks of 1 total in blp.  (1 blwis in pool)
00010000:00010000:2.0:1775765500.170877:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00010000:00010000:1.0:1775765500.170880:0:46896:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9f14b38c6800 lock: ffff9f14ab59b000/0x23cab7dad48e6357 lrc: 2/0,0 mode: PR/PR res: [0x200000405:0x7e:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x9293047872336aa expref: -99 pid: 74538 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0
00010000:00010000:1.0:1775765500.170939:0:46896:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9f14b38c6800 lock: ffff9f14ab59ac00/0x23cab7dad48e6334 lrc: 2/0,0 mode: CR/CR res: [0x200000405:0x7e:0x0].0x0 bits 0xa/0x0 rrc: 3 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x92930478723366b expref: -99 pid: 74118 timeout: 0 lvb_type: 3 lru_score: 4 lru_type: 1
00010000:00010000:1.0:1775765500.170985:0:46896:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9f14b38c6800 lock: ffff9f149153ec00/0x23cab7dad48e6350 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x92930478723368e expref: -99 pid: 74538 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0
00010000:00010000:1.0:1775765500.171030:0:46896:0:(ldlm_request.c:1438:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9f14b38c6800 lock: ffff9f14ab59b000/0x23cab7dad48e6357 lrc: 1/0,0 mode: --/PR res: [0x200000405:0x7e:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x9293047872336aa expref: -99 pid: 74538 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0
00010000:00010000:1.0:1775765500.171038:0:46896:0:(ldlm_request.c:1438:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9f14b38c6800 lock: ffff9f149153ec00/0x23cab7dad48e6350 lrc: 1/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x92930478723368e expref: -99 pid: 74538 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0
00010000:00010000:1.0:1775765500.171053:0:46896:0:(ldlm_request.c:1490:ldlm_cancel_pack()) 2 locks packed
00010000:00010000:1.0:1775765500.171079:0:46896:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9f14b38c6800 lock: ffff9f14ab59b000/0x23cab7dad48e6357 lrc: 0/0,0 mode: --/PR res: [0x200000405:0x7e:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x9293047872336aa expref: -99 pid: 74538 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0
00010000:00010000:1.0:1775765500.171100:0:46896:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9f14b38c6800 lock: ffff9f149153ec00/0x23cab7dad48e6350 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x92930478723368e expref: -99 pid: 74538 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0
00010000:00010000:1.0:1775765500.171118:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00010000:00010000:1.0:1775765500.171120:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00010000:00010000:1.0:1775765500.171123:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1775765500.171145:0:2403:0:(client.c:1830:ptlrpc_send_new_req()) @@@ req waiting for recovery: (FULL != DISCONN)  req@ffff9f14ab4fc700 x1862022114315776/t0(0) o103->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:17/18 lens 328/224 e 0 to 0 dl 0 ref 1 fl Rpc:WQU/200/ffffffff rc 0/-1 job:'ldlm_bl.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765500.171204:0:2403:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0]
00000100:00080000:1.0:1775765500.171218:0:2403:0:(client.c:1830:ptlrpc_send_new_req()) @@@ req waiting for recovery: (FULL != DISCONN)  req@ffff9f14a04d2d80 x1862022114315904/t0(0) o103->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:17/18 lens 328/224 e 0 to 0 dl 0 ref 1 fl Rpc:WQU/200/ffffffff rc 0/-1 job:'ldlm_bl.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775765500.938653:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775765501.194880:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765501.962643:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:1.0:1775765501.962797:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114316032:192.168.202.110@tcp:250:kworker.0
00000800:00000100:1.0:1775765501.962861:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765501.962909:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765501/real 1775765501]  req@ffff9f14a04d1180 x1862022114316032/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765526 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765501.962935:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1180 x1862022114316032/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765526 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765501.962971:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765501.962977:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765501.962986:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114316032:192.168.202.110@tcp:250:kworker.0
00000100:00100000:1.0:1775765501.963013:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114316160:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765501.963040:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765501.963057:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765501/real 1775765501]  req@ffff9f14a04d3800 x1862022114316160/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765526 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765501.963067:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3800 x1862022114316160/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765526 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765501.963073:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765501.963075:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765501.963078:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114316160:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765501.963084:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114316288:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765501.963095:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765501.963101:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765501/real 1775765501]  req@ffff9f14a04d2300 x1862022114316288/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765526 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765501.963108:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d2300 x1862022114316288/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765526 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765501.963125:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765501.963129:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765501.963132:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114316288:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765501.963138:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114316416:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765501.963160:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765501.963171:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765501/real 1775765501]  req@ffff9f14a04d1500 x1862022114316416/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765526 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765501.963183:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1500 x1862022114316416/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765526 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765501.963189:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765501.963191:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765501.963193:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114316416:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775765501.963197:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114316544:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765501.963211:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765501.963218:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765501/real 1775765501]  req@ffff9f14a04d0000 x1862022114316544/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765526 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765501.963225:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0000 x1862022114316544/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765526 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765501.963230:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765501.963232:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765501.963233:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114316544:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775765501.963237:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114316672:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765501.963246:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765501.963251:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765501/real 1775765501]  req@ffff9f14a04d3480 x1862022114316672/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765526 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765501.963259:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3480 x1862022114316672/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765526 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765501.963264:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765501.963265:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765501.963267:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114316672:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765501.963279:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114316800:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765501.963288:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765501.963293:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765501/real 1775765501]  req@ffff9f14a04d0e00 x1862022114316800/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765526 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765501.963300:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0e00 x1862022114316800/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765526 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765501.963309:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765501.963311:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765501.963313:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114316800:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765501.963316:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114316928:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765501.963325:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765501.963330:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765501/real 1775765501]  req@ffff9f14a04d2a00 x1862022114316928/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765526 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765501.963336:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d2a00 x1862022114316928/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765526 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765501.963341:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765501.963343:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765501.963345:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114316928:192.168.202.110@tcp:8:kworker.0
00010000:00010000:1.0:1775765502.218809:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765502.986676:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:2.0:1775765504.010692:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775765504.266845:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765505.034753:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775765505.290898:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765506.058774:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775765506.314769:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765507.082790:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:1.0:1775765507.083105:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114317056:192.168.202.110@tcp:250:kworker.0
00000800:00000100:1.0:1775765507.083201:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765507.083259:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765507/real 1775765507]  req@ffff9f14a04d0a80 x1862022114317056/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765532 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765507.083297:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0a80 x1862022114317056/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765532 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765507.083346:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765507.083356:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765507.083366:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114317056:192.168.202.110@tcp:250:kworker.0
00000100:00100000:1.0:1775765507.083403:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114317184:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765507.083448:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765507.083469:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765507/real 1775765507]  req@ffff9f14a04d0380 x1862022114317184/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765532 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765507.083504:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0380 x1862022114317184/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765532 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765507.083524:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765507.083533:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765507.083547:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114317184:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765507.083633:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114317312:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765507.083696:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765507.083729:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765507/real 1775765507]  req@ffff9f14a04d1f80 x1862022114317312/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765532 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765507.083774:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1f80 x1862022114317312/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765532 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765507.083797:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765507.083807:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765507.083813:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114317312:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765507.083843:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114317440:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765507.083884:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765507.083921:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765507/real 1775765507]  req@ffff9f14a04d3b80 x1862022114317440/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765532 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765507.083945:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3b80 x1862022114317440/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765532 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765507.083960:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765507.083972:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765507.083985:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114317440:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775765507.084003:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114317568:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765507.084039:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765507.084061:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765507/real 1775765507]  req@ffff9f14a04d3100 x1862022114317568/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765532 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765507.084085:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3100 x1862022114317568/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765532 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765507.084099:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765507.084112:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765507.084119:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114317568:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775765507.084142:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114317696:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765507.084176:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765507.084190:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765507/real 1775765507]  req@ffff9f14a04d0a80 x1862022114317696/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765532 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765507.084205:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0a80 x1862022114317696/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765532 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765507.084216:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765507.084219:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765507.084223:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114317696:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765507.084229:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114317824:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765507.084245:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765507.084254:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765507/real 1775765507]  req@ffff9f14a04d2a00 x1862022114317824/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765532 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765507.084267:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d2a00 x1862022114317824/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765532 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765507.084275:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765507.084278:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765507.084282:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114317824:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765507.084293:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114317952:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765507.084309:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765507.084318:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765507/real 1775765507]  req@ffff9f14a04d0e00 x1862022114317952/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765532 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765507.084330:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0e00 x1862022114317952/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765532 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765507.084346:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765507.084350:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765507.084353:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114317952:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775765508.106634:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775765508.362793:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765509.130727:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775765509.386837:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765510.154732:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:2.0:1775765511.178662:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775765511.434826:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1775765512.202648:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765512.202657:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00100000:2.0:1775765512.202666:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775765512.202666:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765512.202677:0:73045:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 2889
00000100:00080000:3.0:1775765512.202687:0:73045:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765512.202694:0:73045:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775765512.202728:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744700 x1862022114318080/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765512.202749:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765512.202752:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775765512.202754:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765512.202757:0:73045:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2889
00000100:00080000:3.0:1775765512.202759:0:73045:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765512.202762:0:73045:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765512.202770:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775765512.202790:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745500 x1862022114318208/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765512.202799:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765512.202802:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775765512.202803:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:1.0:1775765512.202808:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114318080:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775765512.202810:0:73045:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2889
00000100:00080000:3.0:1775765512.202815:0:73045:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765512.202818:0:73045:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765512.202822:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775765512.202838:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745c00 x1862022114318336/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765512.202844:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765512.202847:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775765512.202849:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765512.202852:0:73045:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2889
00000100:00080000:3.0:1775765512.202855:0:73045:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765512.202875:0:73045:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765512.202881:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775765512.202897:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745180 x1862022114318464/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:1.0:1775765512.202903:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775765512.202917:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765512.202922:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775765512.202925:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765512.202933:0:73045:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2889
00000100:00080000:3.0:1775765512.202937:0:73045:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765512.202941:0:73045:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765512.202948:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775765512.202967:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747100 x1862022114318592/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:1.0:1775765512.202968:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765512/real 1775765512]  req@ffff9f1485744700 x1862022114318080/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765537 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775765512.203020:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765512.203023:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775765512.203025:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765512.203029:0:73045:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 2889
00000100:00080000:3.0:1775765512.203032:0:73045:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:1.0:1775765512.203036:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744700 x1862022114318080/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765537 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775765512.203047:0:73045:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765512.203053:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00080000:1.0:1775765512.203066:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00100000:3.0:1775765512.203066:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744000 x1862022114318720/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765512.203077:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:3.0:1775765512.203079:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765512.203084:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775765512.203087:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:1.0:1775765512.203088:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114318080:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775765512.203095:0:73045:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 2889
00000100:00080000:3.0:1775765512.203098:0:73045:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765512.203100:0:73045:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765512.203104:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775765512.203110:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747800 x1862022114318848/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:1.0:1775765512.203115:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114318208:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775765512.203124:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765512.203127:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775765512.203129:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765512.203133:0:73045:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 2889
00000100:00080000:3.0:1775765512.203136:0:73045:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765512.203139:0:73045:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765512.203142:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:1.0:1775765512.203147:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775765512.203152:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745f80 x1862022114318976/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:1.0:1775765512.203163:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765512/real 1775765512]  req@ffff9f1485745500 x1862022114318208/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765537 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765512.203174:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745500 x1862022114318208/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765537 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765512.203180:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765512.203184:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765512.203186:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114318208:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765512.203191:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114318336:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765512.203206:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765512.203214:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765512/real 1775765512]  req@ffff9f1485745c00 x1862022114318336/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765537 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765512.203230:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745c00 x1862022114318336/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765537 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765512.203236:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765512.203240:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765512.203243:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114318336:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765512.203250:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114318464:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765512.203268:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765512.203275:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765512/real 1775765512]  req@ffff9f1485745180 x1862022114318464/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765537 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765512.203286:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745180 x1862022114318464/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765537 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765512.203293:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765512.203296:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765512.203298:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114318464:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775765512.203303:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114318592:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765512.203328:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765512.203347:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765512/real 1775765512]  req@ffff9f1485747100 x1862022114318592/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765537 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765512.203362:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747100 x1862022114318592/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765537 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765512.203371:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765512.203375:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765512.203378:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114318592:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775765512.203385:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114318720:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765512.203405:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765512.203415:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765512/real 1775765512]  req@ffff9f1485744000 x1862022114318720/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765537 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765512.203424:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744000 x1862022114318720/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765537 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765512.203432:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765512.203435:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765512.203437:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114318720:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765512.203455:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114318848:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765512.203467:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765512.203475:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765512/real 1775765512]  req@ffff9f1485747800 x1862022114318848/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765537 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765512.203490:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747800 x1862022114318848/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765537 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765512.203499:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765512.203502:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765512.203505:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114318848:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765512.203509:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114318976:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765512.203521:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765512.203527:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765512/real 1775765512]  req@ffff9f1485745f80 x1862022114318976/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765537 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765512.203535:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745f80 x1862022114318976/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765537 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765512.203540:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765512.203542:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765512.203544:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114318976:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775765512.458782:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765513.226665:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775765513.482824:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765514.250660:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:2.0:1775765515.274749:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:2.0:1775765516.298731:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775765516.554716:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1775765517.322688:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765517.322698:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00100000:2.0:1775765517.322700:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775765517.322702:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765517.322715:0:73045:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 2894
00000100:00080000:3.0:1775765517.322728:0:73045:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765517.322737:0:73045:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775765517.322798:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747480 x1862022114319104/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765517.322843:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765517.322846:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775765517.322848:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765517.322852:0:73045:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2894
00000100:00080000:3.0:1775765517.322856:0:73045:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765517.322859:0:73045:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765517.322870:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775765517.322885:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745f80 x1862022114319232/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765517.322895:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765517.322909:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775765517.322911:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:1.0:1775765517.322915:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114319104:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775765517.322917:0:73045:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2894
00000100:00080000:3.0:1775765517.322925:0:73045:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765517.322929:0:73045:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765517.322933:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775765517.322942:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747800 x1862022114319360/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765517.322956:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765517.322959:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775765517.322961:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765517.322964:0:73045:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2894
00000100:00080000:3.0:1775765517.322967:0:73045:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765517.322970:0:73045:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765517.322973:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775765517.322981:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744000 x1862022114319488/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765517.322988:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765517.322991:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775765517.322993:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000800:00000100:1.0:1775765517.322997:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775765517.323000:0:73045:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2894
00000100:00080000:3.0:1775765517.323004:0:73045:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765517.323007:0:73045:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765517.323011:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775765517.323021:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747100 x1862022114319616/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765517.323028:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765517.323032:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775765517.323033:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765517.323037:0:73045:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 2894
00000100:00080000:3.0:1775765517.323066:0:73045:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:1.0:1775765517.323071:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765517/real 1775765517]  req@ffff9f1485747480 x1862022114319104/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765542 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775765517.323072:0:73045:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765517.323079:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775765517.323092:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745180 x1862022114319744/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765517.323098:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:1.0:1775765517.323102:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747480 x1862022114319104/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765542 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775765517.323102:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775765517.323103:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765517.323123:0:73045:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 2894
00000100:00080000:3.0:1775765517.323127:0:73045:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765517.323133:0:73045:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765517.323137:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00080000:1.0:1775765517.323138:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00100000:3.0:1775765517.323145:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745c00 x1862022114319872/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765517.323146:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765517.323155:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114319104:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775765517.323157:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765517.323164:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775765517.323166:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765517.323169:0:73045:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 2894
00000100:00080000:3.0:1775765517.323171:0:73045:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765517.323174:0:73045:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765517.323176:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775765517.323182:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745500 x1862022114320000/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:1.0:1775765517.323191:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114319232:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765517.323222:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765517.323245:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765517/real 1775765517]  req@ffff9f1485745f80 x1862022114319232/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765542 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765517.323256:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745f80 x1862022114319232/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765542 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765517.323264:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765517.323267:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765517.323270:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114319232:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765517.323275:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114319360:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765517.323294:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765517.323313:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765517/real 1775765517]  req@ffff9f1485747800 x1862022114319360/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765542 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765517.323324:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747800 x1862022114319360/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765542 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765517.323332:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765517.323335:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765517.323353:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114319360:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765517.323361:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114319488:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765517.323385:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765517.323396:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765517/real 1775765517]  req@ffff9f1485744000 x1862022114319488/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765542 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765517.323405:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744000 x1862022114319488/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765542 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765517.323412:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765517.323415:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765517.323417:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114319488:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775765517.323422:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114319616:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765517.323436:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765517.323443:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765517/real 1775765517]  req@ffff9f1485747100 x1862022114319616/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765542 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765517.323451:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747100 x1862022114319616/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765542 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765517.323457:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765517.323459:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765517.323461:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114319616:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775765517.323465:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114319744:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765517.323476:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765517.323482:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765517/real 1775765517]  req@ffff9f1485745180 x1862022114319744/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765542 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765517.323492:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745180 x1862022114319744/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765542 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765517.323498:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765517.323500:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765517.323502:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114319744:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765517.323505:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114319872:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765517.323516:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765517.323522:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765517/real 1775765517]  req@ffff9f1485745c00 x1862022114319872/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765542 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765517.323530:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745c00 x1862022114319872/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765542 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765517.323536:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765517.323538:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765517.323540:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114319872:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765517.323556:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114320000:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765517.323567:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765517.323629:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765517/real 1775765517]  req@ffff9f1485745500 x1862022114320000/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765542 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765517.323638:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745500 x1862022114320000/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765542 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765517.323648:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765517.323650:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765517.323652:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114320000:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775765518.346714:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775765518.602759:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765519.370751:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775765519.626804:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765520.394701:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775765520.650812:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765521.418629:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:2.0:1775765522.442697:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:1.0:1775765522.442963:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114320128:192.168.202.110@tcp:250:kworker.0
00000800:00000100:1.0:1775765522.443163:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765522.443261:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765522/real 1775765522]  req@ffff9f14a04d3480 x1862022114320128/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765547 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765522.443370:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3480 x1862022114320128/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765547 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765522.443431:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765522.443448:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765522.443463:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114320128:192.168.202.110@tcp:250:kworker.0
00000100:00100000:1.0:1775765522.443507:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114320256:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765522.443633:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765522.443653:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765522/real 1775765522]  req@ffff9f14a04d0000 x1862022114320256/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765547 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765522.443667:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0000 x1862022114320256/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765547 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765522.443677:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765522.443682:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765522.443685:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114320256:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765522.443719:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114320384:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765522.443738:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765522.443748:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765522/real 1775765522]  req@ffff9f14a04d1500 x1862022114320384/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765547 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765522.443777:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1500 x1862022114320384/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765547 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765522.443785:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765522.443788:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765522.443791:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114320384:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765522.443797:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114320512:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765522.443813:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765522.443821:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765522/real 1775765522]  req@ffff9f14a04d2300 x1862022114320512/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765547 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765522.443848:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d2300 x1862022114320512/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765547 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765522.443857:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765522.443860:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765522.443863:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114320512:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775765522.443869:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114320640:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765522.443886:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765522.443895:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765522/real 1775765522]  req@ffff9f14a04d3800 x1862022114320640/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765547 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765522.443918:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3800 x1862022114320640/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765547 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765522.443926:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765522.443929:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765522.443932:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114320640:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775765522.443937:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114320768:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765522.443952:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765522.443960:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765522/real 1775765522]  req@ffff9f14a04d1180 x1862022114320768/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765547 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765522.443971:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1180 x1862022114320768/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765547 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765522.443980:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765522.443982:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765522.443985:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114320768:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765522.443991:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114320896:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765522.444028:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765522.444038:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765522/real 1775765522]  req@ffff9f14a04d0e00 x1862022114320896/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765547 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765522.444051:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0e00 x1862022114320896/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765547 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765522.444068:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765522.444072:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765522.444076:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114320896:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765522.444099:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114321024:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765522.444117:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765522.444126:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765522/real 1775765522]  req@ffff9f14a04d2a00 x1862022114321024/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765547 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765522.444144:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d2a00 x1862022114321024/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765547 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765522.444152:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765522.444155:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765522.444158:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114321024:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775765523.466650:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775765523.722740:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765524.490645:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:2.0:1775765525.514639:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775765525.770774:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775765526.538737:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775765526.794735:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775765527.562631:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:2.0:1775765527.562662:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765527.562668:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1775765527.562672:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765527.562681:0:73045:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 2905
00000100:00080000:2.0:1775765527.562691:0:73045:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765527.562697:0:73045:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775765527.562732:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3800 x1862022114321152/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775765527.562759:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765527.562762:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775765527.562768:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765527.562771:0:73045:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2905
00000100:00080000:2.0:1775765527.562774:0:73045:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765527.562777:0:73045:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775765527.562787:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775765527.562797:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0380 x1862022114321280/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775765527.562808:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765527.562811:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775765527.562814:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:1.0:1775765527.562817:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114321152:192.168.202.110@tcp:250:kworker.0
00000100:00080000:2.0:1775765527.562821:0:73045:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2905
00000100:00080000:2.0:1775765527.562828:0:73045:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765527.562832:0:73045:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775765527.562836:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775765527.562854:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0000 x1862022114321408/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775765527.562862:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765527.562865:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775765527.562866:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765527.562869:0:73045:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2905
00000100:00080000:2.0:1775765527.562874:0:73045:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765527.562878:0:73045:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775765527.562884:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:1.0:1775765527.562887:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775765527.562904:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3b80 x1862022114321536/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775765527.562917:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765527.562938:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1775765527.562941:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765527.562948:0:73045:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2905
00000100:00080000:2.0:1775765527.562955:0:73045:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00100000:1.0:1775765527.562957:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765527/real 1775765527]  req@ffff9f14852d3800 x1862022114321152/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765552 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765527.562967:0:73045:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775765527.562977:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775765527.562994:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3100 x1862022114321664/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:1.0:1775765527.563001:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3800 x1862022114321152/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765552 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765527.563012:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765527.563022:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775765527.563025:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765527.563027:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775765527.563032:0:73045:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 2905
00000100:00080000:1.0:1775765527.563038:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:2.0:1775765527.563039:0:73045:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:1.0:1775765527.563048:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114321152:192.168.202.110@tcp:250:kworker.0
00000100:00080000:2.0:1775765527.563050:0:73045:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775765527.563059:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775765527.563070:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2300 x1862022114321792/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:1.0:1775765527.563082:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114321280:192.168.202.110@tcp:38:kworker.0
00000100:00080000:2.0:1775765527.563082:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765527.563093:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775765527.563097:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765527.563101:0:73045:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 2905
00000100:00080000:2.0:1775765527.563104:0:73045:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765527.563108:0:73045:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775765527.563112:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000800:00000100:1.0:1775765527.563122:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775765527.563127:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3480 x1862022114321920/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775765527.563146:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765527.563149:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775765527.563150:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:1.0:1775765527.563153:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765527/real 1775765527]  req@ffff9f14852d0380 x1862022114321280/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765552 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765527.563156:0:73045:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 2905
00000100:00080000:2.0:1775765527.563163:0:73045:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765527.563172:0:73045:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775765527.563179:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775765527.563184:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0380 x1862022114321280/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765552 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775765527.563193:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1c00 x1862022114322048/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765527.563201:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765527.563212:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765527.563217:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114321280:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765527.563224:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114321408:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765527.563243:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765527.563270:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765527/real 1775765527]  req@ffff9f14852d0000 x1862022114321408/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765552 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765527.563279:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0000 x1862022114321408/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765552 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765527.563286:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765527.563288:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765527.563291:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114321408:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765527.563295:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114321536:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765527.563307:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765527.563314:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765527/real 1775765527]  req@ffff9f14852d3b80 x1862022114321536/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765552 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765527.563323:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3b80 x1862022114321536/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765552 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765527.563329:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765527.563331:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765527.563333:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114321536:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775765527.563338:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114321664:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765527.563349:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765527.563355:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765527/real 1775765527]  req@ffff9f14852d3100 x1862022114321664/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765552 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765527.563364:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3100 x1862022114321664/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765552 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765527.563369:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765527.563371:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765527.563381:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114321664:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775765527.563395:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114321792:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765527.563408:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765527.563415:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765527/real 1775765527]  req@ffff9f14852d2300 x1862022114321792/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765552 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765527.563424:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2300 x1862022114321792/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765552 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765527.563431:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765527.563433:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765527.563436:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114321792:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765527.563439:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114321920:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765527.563450:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765527.563456:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765527/real 1775765527]  req@ffff9f14852d3480 x1862022114321920/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765552 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765527.563488:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3480 x1862022114321920/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765552 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765527.563499:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765527.563503:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765527.563507:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114321920:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765527.563515:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114322048:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765527.563540:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765527.563578:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765527/real 1775765527]  req@ffff9f14852d1c00 x1862022114322048/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765552 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765527.563590:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1c00 x1862022114322048/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765552 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765527.563597:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765527.563599:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765527.563602:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114322048:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775765527.818797:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775765528.586629:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:3.0:1775765529.610608:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:2.0:1775765530.634659:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775765530.890768:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765531.658656:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:2.0:1775765532.682688:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765532.682701:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1775765532.682707:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765532.682720:0:73045:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 2910
00000100:00080000:2.0:1775765532.682735:0:73045:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765532.682745:0:73045:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775765532.682745:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:2.0:1775765532.682812:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1880 x1862022114322176/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775765532.682846:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765532.682850:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775765532.682853:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765532.682861:0:73045:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2910
00000100:00080000:2.0:1775765532.682865:0:73045:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765532.682902:0:73045:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775765532.682914:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775765532.682930:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0e00 x1862022114322304/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775765532.682943:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765532.682947:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775765532.682950:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765532.682955:0:73045:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2910
00000100:00080000:2.0:1775765532.682961:0:73045:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765532.682965:0:73045:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775765532.682988:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775765532.683001:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1c00 x1862022114322432/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775765532.683015:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:1.0:1775765532.683019:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114322176:192.168.202.110@tcp:250:kworker.0
00000100:00080000:2.0:1775765532.683020:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775765532.683025:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765532.683036:0:73045:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2910
00000100:00080000:2.0:1775765532.683040:0:73045:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765532.683044:0:73045:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775765532.683050:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775765532.683063:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3480 x1862022114322560/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775765532.683076:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765532.683080:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1775765532.683082:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765532.683088:0:73045:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2910
00000100:00080000:2.0:1775765532.683092:0:73045:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765532.683095:0:73045:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775765532.683100:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775765532.683112:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2300 x1862022114322688/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775765532.683140:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765532.683145:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775765532.683147:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765532.683153:0:73045:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 2910
00000100:00080000:2.0:1775765532.683157:0:73045:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765532.683163:0:73045:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775765532.683171:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775765532.683190:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114322304:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775765532.683191:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3100 x1862022114322816/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775765532.683212:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765532.683217:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775765532.683220:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765532.683225:0:73045:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 2910
00000100:00080000:2.0:1775765532.683232:0:73045:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765532.683237:0:73045:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775765532.683245:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775765532.683272:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3b80 x1862022114322944/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:1.0:1775765532.683287:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114322432:192.168.202.110@tcp:38:kworker.0
00000100:00080000:2.0:1775765532.683288:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765532.683303:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775765532.683305:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765532.683311:0:73045:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 2910
00000100:00080000:2.0:1775765532.683318:0:73045:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:1.0:1775765532.683363:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114322560:192.168.202.110@tcp:8:kworker.0
00000100:00080000:2.0:1775765532.683370:0:73045:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775765532.683379:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775765532.683400:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0000 x1862022114323072/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:1.0:1775765532.683406:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114322688:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775765532.683509:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114322816:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765532.683609:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114322944:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765532.683652:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114323072:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775765532.938794:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775765533.706687:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775765533.962778:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775765534.730680:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775765534.986841:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775765535.754707:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:1.0:1775765535.755165:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765532/real 1775765535]  req@ffff9f14852d1c00 x1862022114322432/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765557 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765535.755256:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1c00 x1862022114322432/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765557 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765535.755282:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765535.755294:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765535.755303:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114322432:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765535.755321:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765532/real 1775765535]  req@ffff9f14852d3480 x1862022114322560/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765557 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765535.755362:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3480 x1862022114322560/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765557 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765535.755379:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765535.755388:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765535.755428:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114322560:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775765535.755446:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765532/real 1775765535]  req@ffff9f14852d2300 x1862022114322688/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765557 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765535.755475:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2300 x1862022114322688/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765557 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765535.755491:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765535.755498:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765535.755504:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114322688:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775765535.755520:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765532/real 1775765535]  req@ffff9f14852d3100 x1862022114322816/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765557 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765535.755543:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3100 x1862022114322816/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765557 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765535.755632:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765535.755644:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765535.755660:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114322816:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765535.755677:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765532/real 1775765535]  req@ffff9f14852d1880 x1862022114322176/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765557 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765535.755706:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1880 x1862022114322176/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765557 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765535.755722:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765535.755730:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765535.755737:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114322176:192.168.202.110@tcp:250:kworker.0
00000100:00100000:1.0:1775765535.755751:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765532/real 1775765535]  req@ffff9f14852d0e00 x1862022114322304/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765557 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765535.755777:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0e00 x1862022114322304/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765557 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765535.755798:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765535.755809:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765535.755819:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114322304:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765535.755905:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765532/real 1775765535]  req@ffff9f14852d3b80 x1862022114322944/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765557 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765535.755956:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3b80 x1862022114322944/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765557 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765535.755974:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765535.755981:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765535.755987:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114322944:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765535.756000:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765532/real 1775765535]  req@ffff9f14852d0000 x1862022114323072/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765557 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765535.756024:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0000 x1862022114323072/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765557 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765535.756039:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765535.756048:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765535.756055:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114323072:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775765536.778681:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:3.0:1775765537.802683:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:2.0:1775765537.802736:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765537.802750:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1775765537.802758:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765537.802774:0:73045:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 2915
00000100:00080000:2.0:1775765537.802791:0:73045:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765537.802802:0:73045:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775765537.802885:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0380 x1862022114323200/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775765537.802941:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765537.802948:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775765537.802959:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765537.802969:0:73045:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2915
00000100:00080000:2.0:1775765537.802994:0:73045:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765537.803003:0:73045:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775765537.803037:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775765537.803084:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114323200:192.168.202.110@tcp:250:kworker.0
00000100:00100000:2.0:1775765537.803084:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3800 x1862022114323328/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775765537.803112:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765537.803118:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775765537.803120:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765537.803126:0:73045:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2915
00000100:00080000:2.0:1775765537.803133:0:73045:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765537.803139:0:73045:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775765537.803148:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775765537.803169:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2d80 x1862022114323456/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775765537.803184:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765537.803189:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775765537.803192:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765537.803198:0:73045:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2915
00000100:00080000:2.0:1775765537.803206:0:73045:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765537.803211:0:73045:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775765537.803221:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:1.0:1775765537.803225:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775765537.803246:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0700 x1862022114323584/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775765537.803264:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765537.803269:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1775765537.803272:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765537.803279:0:73045:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2915
00000100:00080000:2.0:1775765537.803286:0:73045:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765537.803293:0:73045:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775765537.803302:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775765537.803310:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765537/real 1775765537]  req@ffff9f14852d0380 x1862022114323200/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765562 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775765537.803326:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0a80 x1862022114323712/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775765537.803347:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765537.803354:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00100000:1.0:1775765537.803358:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0380 x1862022114323200/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765562 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765537.803359:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765537.803374:0:73045:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 2915
00000100:00080000:2.0:1775765537.803406:0:73045:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765537.803407:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775765537.803420:0:73045:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00080000:1.0:1775765537.803422:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765537.803436:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114323200:192.168.202.110@tcp:250:kworker.0
00000008:00100000:2.0:1775765537.803436:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775765537.803466:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1500 x1862022114323840/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775765537.803493:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:1.0:1775765537.803498:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114323328:192.168.202.110@tcp:38:kworker.0
00000100:00080000:2.0:1775765537.803502:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775765537.803509:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765537.803518:0:73045:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 2915
00000100:00080000:2.0:1775765537.803526:0:73045:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765537.803533:0:73045:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775765537.803543:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000800:00000100:1.0:1775765537.803604:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765537.803631:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765537/real 1775765537]  req@ffff9f14852d3800 x1862022114323328/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765562 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775765537.803640:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2680 x1862022114323968/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775765537.803670:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:1.0:1775765537.803671:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3800 x1862022114323328/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765562 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765537.803686:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775765537.803693:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765537.803711:0:73045:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 2915
00000100:00080000:2.0:1775765537.803724:0:73045:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765537.803726:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775765537.803734:0:73045:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00080000:1.0:1775765537.803744:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000008:00100000:2.0:1775765537.803745:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775765537.803756:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114323328:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775765537.803774:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2a00 x1862022114324096/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:1.0:1775765537.803775:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114323456:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765537.803828:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765537.803850:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765537/real 1775765537]  req@ffff9f14852d2d80 x1862022114323456/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765562 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765537.803872:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2d80 x1862022114323456/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765562 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765537.803886:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765537.803892:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765537.803897:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114323456:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765537.803908:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114323584:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765537.803945:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765537.803964:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765537/real 1775765537]  req@ffff9f14852d0700 x1862022114323584/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765562 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765537.803987:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0700 x1862022114323584/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765562 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765537.804000:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765537.804021:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765537.804027:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114323584:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775765537.804035:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114323712:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765537.804065:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765537.804081:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765537/real 1775765537]  req@ffff9f14852d0a80 x1862022114323712/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765562 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765537.804096:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0a80 x1862022114323712/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765562 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765537.804108:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765537.804112:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765537.804116:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114323712:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775765537.804142:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114323840:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765537.804171:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765537.804189:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765537/real 1775765537]  req@ffff9f14852d1500 x1862022114323840/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765562 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765537.804204:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1500 x1862022114323840/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765562 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765537.804215:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765537.804219:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765537.804224:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114323840:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765537.804232:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114323968:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765537.804258:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765537.804271:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765537/real 1775765537]  req@ffff9f14852d2680 x1862022114323968/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765562 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765537.804285:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2680 x1862022114323968/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765562 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765537.804296:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765537.804300:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765537.804304:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114323968:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765537.804312:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114324096:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765537.804338:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765537.804353:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765537/real 1775765537]  req@ffff9f14852d2a00 x1862022114324096/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765562 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765537.804369:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2a00 x1862022114324096/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765562 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765537.804405:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765537.804410:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765537.804415:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114324096:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775765538.058785:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775765538.826717:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:3.0:1775765539.850646:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775765540.106907:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775765540.874768:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775765541.130860:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775765541.898639:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775765542.154779:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1775765542.922690:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:3.0:1775765542.922699:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:2.0:1775765542.922703:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1775765542.922711:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765542.922725:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 2920
00000100:00080000:2.0:1775765542.922740:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765542.922750:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775765542.922808:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1f80 x1862022114324224/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775765542.922845:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765542.922849:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775765542.922851:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765542.922856:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2920
00000100:00080000:2.0:1775765542.922859:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765542.922862:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775765542.922875:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775765542.922889:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1180 x1862022114324352/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775765542.922901:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:1.0:1775765542.922905:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114324224:192.168.202.110@tcp:250:kworker.0
00000100:00080000:2.0:1775765542.922906:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775765542.922910:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765542.922921:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2920
00000100:00080000:2.0:1775765542.922924:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765542.922927:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775765542.922932:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775765542.922947:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2a00 x1862022114324480/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775765542.922969:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765542.922974:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775765542.922977:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765542.922983:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2920
00000100:00080000:2.0:1775765542.922990:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765542.922996:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775765542.923004:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:1.0:1775765542.923007:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775765542.923043:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2680 x1862022114324608/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775765542.923059:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765542.923065:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1775765542.923069:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765542.923076:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2920
00000100:00100000:1.0:1775765542.923082:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765542/real 1775765542]  req@ffff9f14852d1f80 x1862022114324224/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765567 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765542.923085:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765542.923097:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775765542.923106:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775765542.923119:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1f80 x1862022114324224/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765567 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775765542.923124:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1500 x1862022114324736/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775765542.923170:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765542.923174:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775765542.923175:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775765542.923177:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765542.923189:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 2920
00000100:00080000:1.0:1775765542.923191:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:2.0:1775765542.923195:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:1.0:1775765542.923204:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114324224:192.168.202.110@tcp:250:kworker.0
00000100:00080000:2.0:1775765542.923205:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775765542.923216:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775765542.923228:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0a80 x1862022114324864/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775765542.923236:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765542.923239:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775765542.923240:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765542.923245:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 2920
00000100:00100000:1.0:1775765542.923250:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114324352:192.168.202.110@tcp:38:kworker.0
00000100:00080000:2.0:1775765542.923250:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765542.923259:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775765542.923264:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775765542.923276:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0700 x1862022114324992/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775765542.923286:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765542.923294:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775765542.923296:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765542.923301:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 2920
00000100:00080000:2.0:1775765542.923305:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000800:00000100:1.0:1775765542.923308:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:2.0:1775765542.923309:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775765542.923318:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775765542.923358:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765542/real 1775765542]  req@ffff9f14852d1180 x1862022114324352/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765567 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775765542.923360:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2d80 x1862022114325120/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:1.0:1775765542.923403:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1180 x1862022114324352/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765567 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765542.923433:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765542.923439:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765542.923446:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114324352:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765542.923454:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114324480:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765542.923479:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765542.923489:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765542/real 1775765542]  req@ffff9f14852d2a00 x1862022114324480/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765567 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765542.923501:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2a00 x1862022114324480/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765567 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765542.923509:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765542.923512:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765542.923515:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114324480:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765542.923522:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114324608:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765542.923538:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765542.923546:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765542/real 1775765542]  req@ffff9f14852d2680 x1862022114324608/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765567 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765542.923594:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2680 x1862022114324608/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765567 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765542.923603:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765542.923606:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765542.923609:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114324608:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775765542.923615:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114324736:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765542.923648:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765542.923659:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765542/real 1775765542]  req@ffff9f14852d1500 x1862022114324736/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765567 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765542.923670:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1500 x1862022114324736/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765567 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765542.923678:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765542.923681:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765542.923684:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114324736:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775765542.923689:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114324864:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765542.923705:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765542.923713:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765542/real 1775765542]  req@ffff9f14852d0a80 x1862022114324864/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765567 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765542.923724:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0a80 x1862022114324864/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765567 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765542.923732:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765542.923734:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765542.923737:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114324864:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765542.923756:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114324992:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765542.923772:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765542.923780:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765542/real 1775765542]  req@ffff9f14852d0700 x1862022114324992/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765567 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765542.923792:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0700 x1862022114324992/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765567 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765542.923799:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765542.923801:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765542.923804:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114324992:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765542.923809:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114325120:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765542.923823:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765542.923831:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765542/real 1775765542]  req@ffff9f14852d2d80 x1862022114325120/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765567 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765542.923841:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2d80 x1862022114325120/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765567 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765542.923856:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765542.923859:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765542.923862:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114325120:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775765543.946623:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:3.0:1775765544.970739:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775765545.226835:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775765545.994769:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:3.0:1775765547.018671:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775765547.274827:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775765548.042709:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:2.0:1775765548.042780:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765548.042795:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1775765548.042803:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765548.042820:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 2925
00000100:00080000:2.0:1775765548.042841:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765548.042852:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775765548.042942:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3800 x1862022114325248/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775765548.042987:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765548.042993:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775765548.042996:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765548.043003:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2925
00000100:00080000:2.0:1775765548.043009:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765548.043015:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775765548.043034:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775765548.043067:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0380 x1862022114325376/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775765548.043089:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765548.043111:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775765548.043121:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765548.043142:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2925
00000100:00080000:2.0:1775765548.043148:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765548.043154:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775765548.043161:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775765548.043178:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0000 x1862022114325504/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775765548.043194:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765548.043198:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775765548.043200:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765548.043205:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2925
00000100:00080000:2.0:1775765548.043209:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765548.043213:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775765548.043219:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775765548.043236:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3b80 x1862022114325632/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775765548.043250:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765548.043255:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1775765548.043258:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765548.043264:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2925
00000100:00080000:2.0:1775765548.043272:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765548.043302:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775765548.043315:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775765548.043344:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0e00 x1862022114325760/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775765548.043424:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765548.043436:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775765548.043453:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765548.043474:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 2925
00000100:00080000:2.0:1775765548.043491:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765548.043514:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775765548.043527:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775765548.043551:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1880 x1862022114325888/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775765548.043627:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765548.043636:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775765548.043641:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765548.043654:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 2925
00000100:00080000:2.0:1775765548.043670:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765548.043684:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775765548.043704:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775765548.043751:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3100 x1862022114326016/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775765548.043818:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765548.043824:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775765548.043827:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765548.043843:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 2925
00000100:00080000:2.0:1775765548.043860:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765548.043877:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775765548.043887:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775765548.043919:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2300 x1862022114326144/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00010000:00010000:2.0:1775765548.298807:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775765549.066684:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775765549.322877:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775765550.090728:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775765552.394790:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765553.162724:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:1.0:1775765553.163048:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114326272:192.168.202.110@tcp:250:kworker.0
00000800:00000100:1.0:1775765553.163180:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765553.163307:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765553/real 1775765553]  req@ffff9f14a04d0a80 x1862022114326272/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765578 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765553.163373:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0a80 x1862022114326272/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765578 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765553.163424:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765553.163445:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765553.163461:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114326272:192.168.202.110@tcp:250:kworker.0
00000100:00100000:1.0:1775765553.163496:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114326400:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765553.163645:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765553.163674:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765553/real 1775765553]  req@ffff9f14a04d3100 x1862022114326400/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765578 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765553.163700:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3100 x1862022114326400/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765578 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765553.163716:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765553.163722:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765553.163729:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114326400:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765553.163769:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114326528:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765553.163809:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765553.163830:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765553/real 1775765553]  req@ffff9f14a04d3b80 x1862022114326528/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765578 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765553.163855:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3b80 x1862022114326528/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765578 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765553.163871:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765553.163877:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765553.163883:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114326528:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765553.163894:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114326656:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765553.163943:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765553.163963:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765553/real 1775765553]  req@ffff9f14a04d1f80 x1862022114326656/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765578 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765553.163985:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1f80 x1862022114326656/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765578 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765553.164011:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765553.164017:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765553.164023:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114326656:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775765553.164034:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114326784:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765553.164071:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765553.164092:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765553/real 1775765553]  req@ffff9f14a04d0380 x1862022114326784/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765578 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765553.164117:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0380 x1862022114326784/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765578 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765553.164142:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765553.164148:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765553.164154:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114326784:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775765553.164165:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114326912:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765553.164194:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765553.164208:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765553/real 1775765553]  req@ffff9f14a04d2a00 x1862022114326912/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765578 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765553.164223:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d2a00 x1862022114326912/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765578 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765553.164265:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765553.164270:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765553.164275:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114326912:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765553.164282:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114327040:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765553.164309:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765553.164322:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765553/real 1775765553]  req@ffff9f14a04d0e00 x1862022114327040/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765578 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765553.164337:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0e00 x1862022114327040/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765578 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765553.164355:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765553.164358:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765553.164362:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114327040:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765553.164369:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114327168:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765553.164387:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765553.164396:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765553/real 1775765553]  req@ffff9f14a04d1180 x1862022114327168/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765578 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765553.164409:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1180 x1862022114327168/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765578 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765553.164418:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765553.164422:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765553.164426:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114327168:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775765554.186653:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775765554.442905:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765555.210750:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775765555.466825:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765556.234711:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775765556.490859:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765557.258704:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:2.0:1775765558.282685:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775765558.282685:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765558.282699:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775765558.282704:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765558.282715:0:73045:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 2935
00000100:00080000:3.0:1775765558.282727:0:73045:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765558.282734:0:73045:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775765558.282771:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744700 x1862022114327296/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765558.282811:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765558.282815:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775765558.282817:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765558.282822:0:73045:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2935
00000100:00080000:3.0:1775765558.282826:0:73045:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765558.282832:0:73045:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765558.282842:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775765558.282851:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114327296:192.168.202.110@tcp:250:kworker.0
00000100:00100000:3.0:1775765558.282858:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744380 x1862022114327424/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765558.282870:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765558.282874:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775765558.282876:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765558.282880:0:73045:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2935
00000100:00080000:3.0:1775765558.282885:0:73045:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765558.282889:0:73045:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765558.282894:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775765558.282906:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746a00 x1862022114327552/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:1.0:1775765558.282907:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775765558.282919:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765558.282923:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775765558.282925:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765558.282930:0:73045:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2935
00000100:00080000:3.0:1775765558.282934:0:73045:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765558.282939:0:73045:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775765558.282943:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765558/real 1775765558]  req@ffff9f1485744700 x1862022114327296/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765583 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:3.0:1775765558.282946:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775765558.282962:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744700 x1862022114327296/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765583 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775765558.282966:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746300 x1862022114327680/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765558.282978:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765558.282985:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:3.0:1775765558.282986:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:1.0:1775765558.283004:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114327296:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775765558.283004:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775765558.283008:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765558.283015:0:73045:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2935
00000100:00080000:3.0:1775765558.283019:0:73045:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765558.283023:0:73045:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775765558.283026:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114327424:192.168.202.110@tcp:38:kworker.0
00000008:00100000:3.0:1775765558.283029:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775765558.283045:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747b80 x1862022114327808/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:1.0:1775765558.283049:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775765558.283070:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765558.283075:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775765558.283077:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:1.0:1775765558.283083:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765558/real 1775765558]  req@ffff9f1485744380 x1862022114327424/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765583 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775765558.283083:0:73045:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 2935
00000100:00080000:3.0:1775765558.283091:0:73045:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765558.283099:0:73045:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775765558.283100:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744380 x1862022114327424/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765583 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:3.0:1775765558.283108:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00080000:1.0:1775765558.283110:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765558.283114:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765558.283116:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114327424:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765558.283123:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114327552:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775765558.283124:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744e00 x1862022114327936/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:1.0:1775765558.283143:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775765558.283154:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:1.0:1775765558.283158:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765558/real 1775765558]  req@ffff9f1485746a00 x1862022114327552/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765583 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775765558.283159:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775765558.283162:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765558.283171:0:73045:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 2935
00000100:00080000:3.0:1775765558.283178:0:73045:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765558.283196:0:73045:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775765558.283200:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746a00 x1862022114327552/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765583 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:3.0:1775765558.283203:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00080000:1.0:1775765558.283215:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765558.283219:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775765558.283223:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746680 x1862022114328064/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:1.0:1775765558.283224:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114327552:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765558.283244:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114327680:192.168.202.110@tcp:8:kworker.0
00000100:00080000:3.0:1775765558.283257:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765558.283261:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775765558.283264:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765558.283270:0:73045:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 2935
00000800:00000100:1.0:1775765558.283272:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775765558.283277:0:73045:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765558.283283:0:73045:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775765558.283289:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765558/real 1775765558]  req@ffff9f1485746300 x1862022114327680/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765583 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:3.0:1775765558.283289:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775765558.283305:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745880 x1862022114328192/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:1.0:1775765558.283309:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746300 x1862022114327680/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765583 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765558.283332:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765558.283336:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765558.283341:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114327680:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775765558.283358:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114327808:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765558.283373:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765558.283380:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765558/real 1775765558]  req@ffff9f1485747b80 x1862022114327808/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765583 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765558.283388:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747b80 x1862022114327808/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765583 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765558.283393:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765558.283395:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765558.283397:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114327808:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775765558.283401:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114327936:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765558.283409:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765558.283415:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765558/real 1775765558]  req@ffff9f1485744e00 x1862022114327936/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765583 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765558.283421:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744e00 x1862022114327936/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765583 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765558.283430:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765558.283432:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765558.283434:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114327936:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765558.283437:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114328064:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765558.283446:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765558.283451:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765558/real 1775765558]  req@ffff9f1485746680 x1862022114328064/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765583 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765558.283457:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746680 x1862022114328064/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765583 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765558.283462:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765558.283464:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765558.283465:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114328064:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765558.283468:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114328192:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765558.283477:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765558.283482:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765558/real 1775765558]  req@ffff9f1485745880 x1862022114328192/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765583 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765558.283488:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745880 x1862022114328192/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765583 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765558.283493:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765558.283495:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765558.283497:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114328192:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775765559.306707:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775765559.562853:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765560.330751:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:2.0:1775765561.354661:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775765561.610779:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765562.378646:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775765562.634715:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765563.402716:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775765563.402730:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765563.402740:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775765563.402747:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765563.402761:0:73045:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 2940
00000100:00080000:3.0:1775765563.402782:0:73045:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765563.402797:0:73045:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775765563.402861:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744a80 x1862022114328320/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765563.402893:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765563.402897:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775765563.402899:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765563.402926:0:73045:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2940
00000100:00080000:3.0:1775765563.402932:0:73045:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765563.402936:0:73045:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765563.402952:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775765563.402973:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745880 x1862022114328448/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:1.0:1775765563.402989:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114328320:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775765563.402992:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765563.403006:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775765563.403009:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765563.403015:0:73045:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2940
00000100:00080000:3.0:1775765563.403019:0:73045:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765563.403024:0:73045:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765563.403031:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775765563.403071:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746680 x1862022114328576/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765563.403088:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765563.403095:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775765563.403098:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000800:00000100:1.0:1775765563.403114:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775765563.403148:0:73045:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2940
00000100:00080000:3.0:1775765563.403157:0:73045:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765563.403164:0:73045:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765563.403174:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775765563.403200:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744e00 x1862022114328704/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:1.0:1775765563.403213:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765563/real 1775765563]  req@ffff9f1485744a80 x1862022114328320/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765588 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775765563.403214:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765563.403222:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775765563.403224:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765563.403229:0:73045:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2940
00000100:00080000:3.0:1775765563.403232:0:73045:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765563.403235:0:73045:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765563.403248:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775765563.403270:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747b80 x1862022114328832/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765563.403284:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:1.0:1775765563.403293:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744a80 x1862022114328320/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765588 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775765563.403294:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775765563.403301:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765563.403318:0:73045:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 2940
00000100:00080000:3.0:1775765563.403333:0:73045:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765563.403342:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775765563.403342:0:73045:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765563.403360:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00080000:1.0:1775765563.403366:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765563.403381:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114328320:192.168.202.110@tcp:250:kworker.0
00000100:00100000:3.0:1775765563.403388:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746300 x1862022114328960/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765563.403408:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765563.403412:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775765563.403414:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765563.403420:0:73045:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 2940
00000100:00080000:3.0:1775765563.403422:0:73045:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:1.0:1775765563.403446:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114328448:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775765563.403447:0:73045:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765563.403468:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775765563.403497:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746a00 x1862022114329088/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:1.0:1775765563.403501:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765563.403529:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765563/real 1775765563]  req@ffff9f1485745880 x1862022114328448/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765588 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775765563.403533:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765563.403555:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775765563.403558:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765563.403567:0:73045:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 2940
00000100:00080000:3.0:1775765563.403627:0:73045:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:1.0:1775765563.403633:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745880 x1862022114328448/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765588 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775765563.403638:0:73045:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765563.403658:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00080000:1.0:1775765563.403663:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765563.403670:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765563.403682:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114328448:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775765563.403682:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744380 x1862022114329216/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:1.0:1775765563.403726:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114328576:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765563.403781:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765563.403806:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765563/real 1775765563]  req@ffff9f1485746680 x1862022114328576/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765588 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765563.403831:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746680 x1862022114328576/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765588 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765563.403843:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765563.403847:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765563.403852:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114328576:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765563.403861:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114328704:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765563.403900:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765563.403914:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765563/real 1775765563]  req@ffff9f1485744e00 x1862022114328704/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765588 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765563.403929:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744e00 x1862022114328704/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765588 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765563.403939:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765563.403943:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765563.403947:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114328704:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775765563.403954:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114328832:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765563.403971:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765563.403982:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765563/real 1775765563]  req@ffff9f1485747b80 x1862022114328832/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765588 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765563.403995:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747b80 x1862022114328832/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765588 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765563.404005:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765563.404009:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765563.404013:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114328832:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775765563.404020:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114328960:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765563.404038:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765563.404047:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765563/real 1775765563]  req@ffff9f1485746300 x1862022114328960/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765588 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765563.404060:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746300 x1862022114328960/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765588 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765563.404069:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765563.404073:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765563.404077:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114328960:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765563.404103:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114329088:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765563.404122:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765563.404162:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765563/real 1775765563]  req@ffff9f1485746a00 x1862022114329088/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765588 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765563.404180:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746a00 x1862022114329088/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765588 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765563.404190:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765563.404194:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765563.404199:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114329088:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765563.404207:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114329216:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765563.404234:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765563.404247:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765563/real 1775765563]  req@ffff9f1485744380 x1862022114329216/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765588 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765563.404261:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744380 x1862022114329216/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765588 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765563.404270:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765563.404274:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765563.404278:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114329216:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775765563.658714:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765564.426689:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775765564.682705:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765565.450639:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:2.0:1775765566.474635:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775765566.730657:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765567.498795:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:2.0:1775765568.522630:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775765568.522649:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765568.522654:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775765568.522659:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765568.522666:0:73045:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 2946
00000100:00080000:3.0:1775765568.522675:0:73045:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765568.522680:0:73045:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775765568.522713:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744700 x1862022114329344/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765568.522735:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765568.522738:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775765568.522739:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765568.522742:0:73045:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2946
00000100:00080000:3.0:1775765568.522744:0:73045:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765568.522746:0:73045:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765568.522754:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775765568.522771:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745500 x1862022114329472/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765568.522777:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765568.522780:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775765568.522781:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765568.522783:0:73045:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2946
00000100:00080000:3.0:1775765568.522785:0:73045:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765568.522788:0:73045:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765568.522791:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775765568.522797:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745c00 x1862022114329600/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765568.522807:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765568.522809:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775765568.522811:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765568.522813:0:73045:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2946
00000100:00080000:3.0:1775765568.522815:0:73045:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00100000:1.0:1775765568.522820:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114329344:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775765568.522820:0:73045:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765568.522825:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775765568.522832:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745180 x1862022114329728/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765568.522838:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765568.522840:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775765568.522841:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765568.522844:0:73045:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2946
00000100:00080000:3.0:1775765568.522846:0:73045:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765568.522849:0:73045:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765568.522852:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775765568.522859:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747100 x1862022114329856/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765568.522864:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765568.522867:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775765568.522868:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765568.522871:0:73045:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 2946
00000100:00080000:3.0:1775765568.522874:0:73045:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000800:00000100:1.0:1775765568.522878:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775765568.522881:0:73045:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765568.522885:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775765568.522904:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744000 x1862022114329984/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765568.522910:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765568.522929:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775765568.522930:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765568.522933:0:73045:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 2946
00000100:00080000:3.0:1775765568.522937:0:73045:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765568.522941:0:73045:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765568.522946:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775765568.522949:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765568/real 1775765568]  req@ffff9f1485744700 x1862022114329344/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765593 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775765568.522961:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747800 x1862022114330112/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765568.522970:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765568.522973:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775765568.522975:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765568.523001:0:73045:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 2946
00000100:00080000:3.0:1775765568.523004:0:73045:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:1.0:1775765568.523006:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744700 x1862022114329344/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765593 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775765568.523010:0:73045:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765568.523016:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00080000:1.0:1775765568.523020:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765568.523026:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775765568.523026:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745f80 x1862022114330240/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:1.0:1775765568.523033:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114329344:192.168.202.110@tcp:250:kworker.0
00000100:00100000:1.0:1775765568.523045:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114329472:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765568.523074:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765568.523103:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765568/real 1775765568]  req@ffff9f1485745500 x1862022114329472/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765593 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765568.523112:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745500 x1862022114329472/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765593 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765568.523118:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765568.523120:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765568.523122:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114329472:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765568.523126:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114329600:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765568.523152:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765568.523158:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765568/real 1775765568]  req@ffff9f1485745c00 x1862022114329600/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765593 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765568.523165:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745c00 x1862022114329600/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765593 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765568.523170:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765568.523172:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765568.523174:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114329600:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765568.523191:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114329728:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765568.523202:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765568.523207:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765568/real 1775765568]  req@ffff9f1485745180 x1862022114329728/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765593 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765568.523219:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745180 x1862022114329728/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765593 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765568.523224:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765568.523226:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765568.523228:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114329728:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775765568.523233:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114329856:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765568.523249:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765568.523258:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765568/real 1775765568]  req@ffff9f1485747100 x1862022114329856/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765593 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765568.523271:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747100 x1862022114329856/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765593 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765568.523279:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765568.523283:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765568.523286:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114329856:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775765568.523292:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114329984:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765568.523310:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765568.523318:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765568/real 1775765568]  req@ffff9f1485744000 x1862022114329984/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765593 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765568.523326:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744000 x1862022114329984/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765593 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765568.523335:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765568.523337:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765568.523339:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114329984:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765568.523342:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114330112:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765568.523352:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765568.523358:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765568/real 1775765568]  req@ffff9f1485747800 x1862022114330112/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765593 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765568.523365:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747800 x1862022114330112/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765593 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765568.523369:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765568.523371:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765568.523373:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114330112:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765568.523376:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114330240:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765568.523396:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765568.523401:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765568/real 1775765568]  req@ffff9f1485745f80 x1862022114330240/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765593 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765568.523408:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745f80 x1862022114330240/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765593 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765568.523413:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765568.523414:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765568.523417:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114330240:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775765568.778786:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775765569.546809:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775765569.802752:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775765570.570673:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775765570.826713:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765571.594788:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775765571.850730:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765572.618640:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775765573.642710:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:2.0:1775765573.642722:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775765573.642724:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775765573.642758:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765573.642772:0:73045:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 2951
00000100:00080000:3.0:1775765573.642787:0:73045:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765573.642797:0:73045:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775765573.642871:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747480 x1862022114330368/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765573.642918:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765573.642923:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775765573.642925:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765573.642930:0:73045:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2951
00000100:00080000:3.0:1775765573.642935:0:73045:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765573.642941:0:73045:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765573.642954:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775765573.642976:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745f80 x1862022114330496/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:1.0:1775765573.642978:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114330368:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775765573.642990:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765573.642999:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775765573.643012:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765573.643017:0:73045:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2951
00000100:00080000:3.0:1775765573.643021:0:73045:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765573.643026:0:73045:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765573.643031:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775765573.643063:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747800 x1862022114330624/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765573.643074:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765573.643078:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775765573.643080:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765573.643085:0:73045:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2951
00000100:00080000:3.0:1775765573.643089:0:73045:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765573.643093:0:73045:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765573.643098:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775765573.643112:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744000 x1862022114330752/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765573.643121:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000800:00000100:1.0:1775765573.643129:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775765573.643130:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775765573.643134:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765573.643141:0:73045:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2951
00000100:00080000:3.0:1775765573.643144:0:73045:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765573.643147:0:73045:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765573.643151:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775765573.643162:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747100 x1862022114330880/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765573.643178:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765573.643182:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775765573.643184:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765573.643188:0:73045:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 2951
00000100:00080000:3.0:1775765573.643191:0:73045:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765573.643195:0:73045:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765573.643199:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775765573.643211:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745180 x1862022114331008/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:1.0:1775765573.643219:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765573/real 1775765573]  req@ffff9f1485747480 x1862022114330368/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765598 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775765573.643223:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765573.643234:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775765573.643238:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765573.643250:0:73045:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 2951
00000100:00080000:3.0:1775765573.643258:0:73045:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765573.643261:0:73045:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765573.643266:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775765573.643278:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747480 x1862022114330368/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765598 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775765573.643280:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745c00 x1862022114331136/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765573.643310:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765573.643322:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775765573.643324:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765573.643328:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775765573.643329:0:73045:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 2951
00000100:00080000:3.0:1775765573.643336:0:73045:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765573.643344:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:3.0:1775765573.643344:0:73045:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765573.643352:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775765573.643361:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114330368:192.168.202.110@tcp:250:kworker.0
00000100:00100000:3.0:1775765573.643367:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745500 x1862022114331264/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:1.0:1775765573.643424:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114330496:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765573.643468:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765573.643482:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765573/real 1775765573]  req@ffff9f1485745f80 x1862022114330496/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765598 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765573.643496:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745f80 x1862022114330496/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765598 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765573.643505:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765573.643508:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765573.643512:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114330496:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765573.643517:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114330624:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765573.643536:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765573.643545:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765573/real 1775765573]  req@ffff9f1485747800 x1862022114330624/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765598 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765573.643587:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747800 x1862022114330624/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765598 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765573.643595:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765573.643598:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765573.643601:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114330624:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765573.643607:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114330752:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765573.643627:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765573.643636:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765573/real 1775765573]  req@ffff9f1485744000 x1862022114330752/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765598 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765573.643647:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744000 x1862022114330752/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765598 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765573.643655:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765573.643659:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765573.643661:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114330752:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775765573.643666:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114330880:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765573.643680:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765573.643688:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765573/real 1775765573]  req@ffff9f1485747100 x1862022114330880/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765598 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765573.643705:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747100 x1862022114330880/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765598 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765573.643712:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765573.643720:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765573.643723:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114330880:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775765573.643728:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114331008:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765573.643742:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765573.643750:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765573/real 1775765573]  req@ffff9f1485745180 x1862022114331008/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765598 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765573.643761:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745180 x1862022114331008/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765598 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765573.643768:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765573.643771:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765573.643773:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114331008:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765573.643778:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114331136:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765573.643800:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765573.643807:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765573/real 1775765573]  req@ffff9f1485745c00 x1862022114331136/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765598 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765573.643817:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745c00 x1862022114331136/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765598 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765573.643830:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765573.643833:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765573.643836:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114331136:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765573.643840:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114331264:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765573.643854:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765573.643861:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765573/real 1775765573]  req@ffff9f1485745500 x1862022114331264/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765598 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765573.643870:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745500 x1862022114331264/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765598 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765573.643877:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765573.643880:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765573.643883:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114331264:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775765573.898763:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765574.666721:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:2.0:1775765575.690665:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775765575.946765:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765576.714641:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775765576.970822:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765577.738686:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775765577.994821:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765578.762716:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:1.0:1775765578.763104:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114331392:192.168.202.110@tcp:250:kworker.0
00000800:00000100:1.0:1775765578.763230:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765578.763299:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765578/real 1775765578]  req@ffff9f14a04d3800 x1862022114331392/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765603 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765578.763351:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3800 x1862022114331392/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765603 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765578.763382:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765578.763392:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765578.763407:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114331392:192.168.202.110@tcp:250:kworker.0
00000100:00100000:1.0:1775765578.763432:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114331520:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765578.763500:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765578.763535:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765578/real 1775765578]  req@ffff9f14a04d2300 x1862022114331520/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765603 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765578.763620:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d2300 x1862022114331520/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765603 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765578.763632:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765578.763636:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765578.763640:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114331520:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765578.763678:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114331648:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765578.763708:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765578.763752:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765578/real 1775765578]  req@ffff9f14a04d1500 x1862022114331648/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765603 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765578.763767:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1500 x1862022114331648/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765603 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765578.763777:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765578.763781:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765578.763785:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114331648:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765578.763810:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114331776:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765578.763832:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765578.763842:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765578/real 1775765578]  req@ffff9f14a04d0000 x1862022114331776/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765603 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765578.763856:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0000 x1862022114331776/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765603 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765578.763865:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765578.763868:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765578.763871:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114331776:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775765578.763878:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114331904:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765578.763913:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765578.763923:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765578/real 1775765578]  req@ffff9f14a04d3480 x1862022114331904/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765603 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765578.763937:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3480 x1862022114331904/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765603 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765578.763947:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765578.763950:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765578.763954:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114331904:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775765578.763960:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114332032:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765578.763979:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765578.764036:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765578/real 1775765578]  req@ffff9f14a04d1180 x1862022114332032/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765603 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765578.764048:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1180 x1862022114332032/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765603 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765578.764064:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765578.764067:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765578.764071:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114332032:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765578.764077:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114332160:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765578.764098:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765578.764108:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765578/real 1775765578]  req@ffff9f14a04d0e00 x1862022114332160/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765603 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765578.764120:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0e00 x1862022114332160/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765603 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765578.764127:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765578.764130:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765578.764133:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114332160:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765578.764138:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114332288:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765578.764154:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765578.764163:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765578/real 1775765578]  req@ffff9f14a04d2a00 x1862022114332288/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765603 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765578.764174:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d2a00 x1862022114332288/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765603 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765578.764182:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765578.764185:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765578.764188:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114332288:192.168.202.110@tcp:8:kworker.0
00010000:00010000:1.0:1775765579.018889:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765579.786659:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775765580.042717:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765580.810617:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775765581.066821:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765581.834660:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:3.0:1775765582.858762:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775765583.114756:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1775765583.882667:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765583.882676:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775765583.882681:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765583.882690:0:73045:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 2961
00000100:00080000:3.0:1775765583.882701:0:73045:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765583.882709:0:73045:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775765583.882760:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744700 x1862022114332416/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765583.882796:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765583.882799:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775765583.882801:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765583.882805:0:73045:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2961
00000100:00080000:3.0:1775765583.882810:0:73045:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765583.882814:0:73045:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765583.882824:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775765583.882841:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744380 x1862022114332544/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765583.882855:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765583.882861:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775765583.882879:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765583.882883:0:73045:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2961
00000100:00080000:3.0:1775765583.882888:0:73045:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765583.882894:0:73045:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765583.882899:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775765583.882916:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746a00 x1862022114332672/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765583.882950:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765583.882953:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775765583.882955:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765583.882960:0:73045:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2961
00000100:00080000:3.0:1775765583.882966:0:73045:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765583.882974:0:73045:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765583.882982:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775765583.883002:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746300 x1862022114332800/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765583.883259:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765583.883271:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775765583.883276:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765583.883304:0:73045:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2961
00000100:00080000:3.0:1775765583.883357:0:73045:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765583.883362:0:73045:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765583.883372:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775765583.883395:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747b80 x1862022114332928/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765583.883422:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765583.883427:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775765583.883431:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765583.883437:0:73045:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 2961
00000100:00080000:3.0:1775765583.883441:0:73045:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765583.883444:0:73045:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765583.883448:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775765583.883462:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744e00 x1862022114333056/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765583.883471:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765583.883477:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775765583.883479:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765583.883484:0:73045:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 2961
00000100:00080000:3.0:1775765583.883487:0:73045:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765583.883492:0:73045:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765583.883502:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775765583.883520:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746680 x1862022114333184/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765583.883548:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765583.883583:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775765583.883588:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765583.883595:0:73045:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 2961
00000100:00080000:3.0:1775765583.883602:0:73045:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765583.883609:0:73045:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765583.883617:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775765583.883634:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745880 x1862022114333312/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00010000:00010000:2.0:1775765584.138819:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00010000:00010000:2.0:1775765585.162685:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00010000:00010000:1.0:1775765586.186785:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00010000:00010000:1.0:1775765587.210677:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00010000:00010000:2.0:1775765588.234743:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1775765589.002663:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765589.002681:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775765589.002685:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765589.002706:0:73045:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 2966
00000100:00080000:3.0:1775765589.002718:0:73045:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765589.002747:0:73045:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775765589.002800:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744a80 x1862022114333440/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765589.002826:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765589.002829:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775765589.002847:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765589.002851:0:73045:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2966
00000100:00080000:3.0:1775765589.002855:0:73045:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765589.002858:0:73045:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765589.002867:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775765589.002880:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745880 x1862022114333568/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765589.002947:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765589.002952:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775765589.002954:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765589.002961:0:73045:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2966
00000100:00080000:3.0:1775765589.002983:0:73045:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765589.002988:0:73045:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765589.002994:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775765589.003008:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746680 x1862022114333696/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765589.003023:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765589.003026:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775765589.003027:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765589.003031:0:73045:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2966
00000100:00080000:3.0:1775765589.003063:0:73045:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765589.003072:0:73045:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765589.003078:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775765589.003093:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744e00 x1862022114333824/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765589.003107:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765589.003110:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775765589.003114:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765589.003124:0:73045:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2966
00000100:00080000:3.0:1775765589.003130:0:73045:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765589.003138:0:73045:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765589.003144:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775765589.003164:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747b80 x1862022114333952/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765589.003202:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765589.003209:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775765589.003211:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765589.003217:0:73045:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 2966
00000100:00080000:3.0:1775765589.003223:0:73045:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765589.003230:0:73045:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765589.003236:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775765589.003249:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746300 x1862022114334080/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765589.003271:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765589.003276:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775765589.003279:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765589.003285:0:73045:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 2966
00000100:00080000:3.0:1775765589.003293:0:73045:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765589.003300:0:73045:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765589.003310:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775765589.003328:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746a00 x1862022114334208/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765589.003350:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765589.003358:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775765589.003360:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765589.003366:0:73045:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 2966
00000100:00080000:3.0:1775765589.003376:0:73045:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765589.003384:0:73045:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765589.003395:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775765589.003415:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744380 x1862022114334336/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00010000:00010000:2.0:1775765590.282713:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00010000:00010000:2.0:1775765591.306745:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00010000:00010000:2.0:1775765592.330703:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00010000:00010000:1.0:1775765593.354749:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1775765594.122798:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765594.122810:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1775765594.122816:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765594.122831:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 2971
00000100:00080000:2.0:1775765594.122876:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765594.122888:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775765594.122944:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3480 x1862022114334464/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775765594.122982:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765594.122987:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775765594.122989:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765594.122995:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2971
00000100:00080000:2.0:1775765594.123001:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765594.123007:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775765594.123019:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775765594.123035:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1c00 x1862022114334592/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775765594.123050:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765594.123056:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775765594.123063:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765594.123090:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2971
00000100:00080000:2.0:1775765594.123093:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765594.123097:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775765594.123103:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775765594.123115:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2300 x1862022114334720/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775765594.123126:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765594.123130:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775765594.123131:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765594.123135:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2971
00000100:00080000:2.0:1775765594.123150:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765594.123154:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775765594.123159:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775765594.123172:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3100 x1862022114334848/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775765594.123183:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765594.123186:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1775765594.123188:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765594.123192:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2971
00000100:00080000:2.0:1775765594.123196:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765594.123200:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775765594.123206:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775765594.123220:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1880 x1862022114334976/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775765594.123252:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765594.123255:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775765594.123259:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765594.123270:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 2971
00000100:00080000:2.0:1775765594.123280:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765594.123288:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775765594.123293:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775765594.123304:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0e00 x1862022114335104/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775765594.123329:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765594.123333:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775765594.123336:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765594.123341:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 2971
00000100:00080000:2.0:1775765594.123344:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765594.123348:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775765594.123353:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775765594.123367:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3b80 x1862022114335232/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775765594.123379:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765594.123384:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775765594.123389:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765594.123400:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 2971
00000100:00080000:2.0:1775765594.123404:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765594.123408:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775765594.123413:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775765594.123429:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0000 x1862022114335360/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00010000:00010000:1.0:1775765594.378902:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00010000:00010000:2.0:1775765595.402732:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00010000:00010000:2.0:1775765597.450683:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00010000:00010000:2.0:1775765598.474767:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1775765599.242713:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765599.242720:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775765599.242724:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765599.242734:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 2976
00000100:00080000:3.0:1775765599.242748:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765599.242755:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775765599.242820:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744700 x1862022114335488/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765599.242848:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765599.242850:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775765599.242852:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765599.242855:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2976
00000100:00080000:3.0:1775765599.242858:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765599.242862:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765599.242874:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775765599.242890:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745500 x1862022114335616/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765599.242902:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765599.242905:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775765599.242906:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765599.242909:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2976
00000100:00080000:3.0:1775765599.242911:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765599.242914:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765599.242918:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775765599.242927:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745c00 x1862022114335744/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765599.242934:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765599.242937:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775765599.242938:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765599.242941:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2976
00000100:00080000:3.0:1775765599.242944:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765599.242947:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765599.242952:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775765599.242961:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745180 x1862022114335872/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765599.242968:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765599.242971:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775765599.242973:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765599.242976:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2976
00000100:00080000:3.0:1775765599.242979:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765599.242982:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765599.242986:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775765599.242995:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747100 x1862022114336000/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765599.243002:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765599.243005:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775765599.243006:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765599.243010:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 2976
00000100:00080000:3.0:1775765599.243014:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765599.244895:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765599.244912:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775765599.244948:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744000 x1862022114336128/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765599.244982:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765599.244986:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775765599.244989:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765599.244994:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 2976
00000100:00080000:3.0:1775765599.245004:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765599.245008:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765599.245013:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775765599.245024:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747800 x1862022114336256/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765599.245034:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765599.245038:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775765599.245040:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765599.245046:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 2976
00000100:00080000:3.0:1775765599.245050:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765599.245054:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765599.245058:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775765599.245067:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745f80 x1862022114336384/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00010000:00010000:2.0:1775765599.498881:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00010000:00010000:1.0:1775765600.522743:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00010000:00010000:1.0:1775765601.546744:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000400:00000100:1.0:1775765602.314981:0:2391:0:(lib-socket.c:454:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.202.110:988
00000400:00000100:1.0:1775765602.315033:0:2391:0:(acceptor.c:111:lnet_connect_console_error()) Connection to 192.168.202.110@tcp at host 192.168.202.110 was unreachable: the network or that node may be down, or Lustre may be misconfigured.
00000800:00000100:1.0:1775765602.315053:0:2391:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:1.0:1775765602.315102:0:2391:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:1.0:1775765602.315117:0:2391:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:1.0:1775765602.315135:0:2391:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:1.0:1775765602.315151:0:2391:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:1.0:1775765602.315179:0:2391:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:1.0:1775765602.315198:0:2391:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:1.0:1775765602.315218:0:2391:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00010000:00010000:2.0:1775765602.570718:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765604.362672:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:2.0:1775765604.362713:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765604.362718:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1775765604.362721:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765604.362729:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 2981
00000100:00080000:2.0:1775765604.362738:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765604.362759:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775765604.362805:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0380 x1862022114336512/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775765604.362827:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765604.362830:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775765604.362831:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765604.362834:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2981
00000100:00080000:2.0:1775765604.362837:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765604.362840:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775765604.362849:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775765604.362861:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3800 x1862022114336640/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775765604.362876:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765604.362879:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775765604.362880:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765604.362883:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2981
00000100:00080000:2.0:1775765604.362886:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765604.362888:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775765604.362891:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775765604.362901:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2d80 x1862022114336768/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775765604.362908:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765604.362910:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775765604.362911:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765604.362913:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2981
00000100:00080000:2.0:1775765604.362917:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765604.362919:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775765604.362922:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775765604.362932:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0700 x1862022114336896/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775765604.362939:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765604.362942:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1775765604.362944:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765604.362947:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2981
00000100:00080000:2.0:1775765604.362949:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765604.362951:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775765604.362954:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775765604.362962:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0a80 x1862022114337024/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775765604.362986:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765604.362992:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775765604.362993:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765604.362997:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 2981
00000100:00080000:2.0:1775765604.362999:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765604.363004:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775765604.363011:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775765604.363024:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1500 x1862022114337152/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775765604.363035:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765604.363039:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775765604.363042:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765604.363049:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 2981
00000100:00080000:2.0:1775765604.363054:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765604.363058:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775765604.363063:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775765604.363077:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2680 x1862022114337280/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775765604.363093:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765604.363096:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775765604.363098:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765604.363102:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 2981
00000100:00080000:2.0:1775765604.363108:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765604.363112:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775765604.363118:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775765604.363132:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2a00 x1862022114337408/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00010000:00010000:2.0:1775765604.618731:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765605.386707:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775765605.642775:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765606.410634:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775765606.666688:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765607.434620:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775765607.690744:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765608.458808:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775765608.714886:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1775765609.482777:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:2.0:1775765609.482783:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775765609.482795:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775765609.482801:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765609.482815:0:73045:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 2986
00000100:00080000:3.0:1775765609.482829:0:73045:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765609.482838:0:73045:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775765609.482885:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747480 x1862022114337536/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765609.482920:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765609.482925:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775765609.482928:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765609.482933:0:73045:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2986
00000100:00080000:3.0:1775765609.482937:0:73045:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765609.482943:0:73045:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765609.482955:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775765609.482973:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747800 x1862022114337664/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765609.482987:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765609.482992:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775765609.482994:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765609.482999:0:73045:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2986
00000100:00080000:3.0:1775765609.483005:0:73045:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765609.483014:0:73045:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765609.483028:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775765609.483043:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744000 x1862022114337792/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765609.483056:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765609.483061:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775765609.483063:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765609.483068:0:73045:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2986
00000100:00080000:3.0:1775765609.483074:0:73045:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765609.483077:0:73045:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765609.483083:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775765609.483098:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745f80 x1862022114337920/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765609.483132:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765609.483140:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775765609.483143:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765609.483156:0:73045:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2986
00000100:00080000:3.0:1775765609.483166:0:73045:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765609.483175:0:73045:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765609.483185:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775765609.483215:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744700 x1862022114338048/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765609.483274:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765609.483284:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775765609.483290:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765609.483308:0:73045:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 2986
00000100:00080000:3.0:1775765609.483321:0:73045:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765609.483332:0:73045:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765609.483348:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775765609.483371:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747100 x1862022114338176/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765609.483384:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765609.483395:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775765609.483399:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765609.483407:0:73045:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 2986
00000100:00080000:3.0:1775765609.483413:0:73045:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765609.483422:0:73045:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765609.483435:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775765609.483464:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745180 x1862022114338304/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765609.483504:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765609.483513:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775765609.483521:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765609.483533:0:73045:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 2986
00000100:00080000:3.0:1775765609.483543:0:73045:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765609.483548:0:73045:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765609.483595:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775765609.483619:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745c00 x1862022114338432/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00010000:00010000:2.0:1775765609.738763:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765610.506730:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:2.0:1775765611.530730:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775765611.786846:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765612.554699:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775765612.810836:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765613.578735:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775765613.834802:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765614.602626:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:1.0:1775765614.602658:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765614.602664:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1775765614.602668:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765614.602699:0:73045:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 2992
00000100:00080000:1.0:1775765614.602708:0:73045:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765614.602719:0:73045:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775765614.602752:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdf80 x1862022114338560/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765614.602773:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765614.602776:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775765614.602777:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765614.602780:0:73045:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2992
00000100:00080000:1.0:1775765614.602782:0:73045:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765614.602785:0:73045:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765614.602792:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775765614.602800:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fd880 x1862022114338688/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765614.602808:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765614.602811:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775765614.602812:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765614.602814:0:73045:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2992
00000100:00080000:1.0:1775765614.602816:0:73045:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765614.602821:0:73045:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765614.602825:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775765614.602835:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fce00 x1862022114338816/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765614.602841:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765614.602843:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775765614.602844:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765614.602874:0:73045:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2992
00000100:00080000:1.0:1775765614.602877:0:73045:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765614.602880:0:73045:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765614.602884:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775765614.602899:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdc00 x1862022114338944/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765614.602905:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765614.602909:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1775765614.602912:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765614.602919:0:73045:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2992
00000100:00080000:1.0:1775765614.602922:0:73045:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765614.602925:0:73045:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765614.602928:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775765614.602937:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ffb80 x1862022114339072/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765614.602943:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765614.602946:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775765614.602948:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765614.602951:0:73045:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 2992
00000100:00080000:1.0:1775765614.602953:0:73045:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765614.602959:0:73045:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765614.602965:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775765614.602976:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff800 x1862022114339200/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765614.602988:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765614.602991:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775765614.602994:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765614.603001:0:73045:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 2992
00000100:00080000:1.0:1775765614.603007:0:73045:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765614.603013:0:73045:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765614.603016:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775765614.603025:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fed80 x1862022114339328/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765614.603041:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765614.603044:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775765614.603046:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765614.603050:0:73045:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 2992
00000100:00080000:1.0:1775765614.603052:0:73045:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765614.603054:0:73045:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765614.603058:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775765614.603089:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fca80 x1862022114339456/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00010000:00010000:1.0:1775765614.858760:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765615.626745:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775765615.882740:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765616.650686:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775765616.906699:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765617.674732:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:2.0:1775765618.698686:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775765618.954787:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765619.722665:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775765619.722665:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765619.722673:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775765619.722677:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765619.722690:0:73045:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 2997
00000100:00080000:3.0:1775765619.722701:0:73045:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765619.722709:0:73045:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775765619.722744:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745500 x1862022114339584/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765619.722779:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765619.722783:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775765619.722785:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765619.722790:0:73045:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2997
00000100:00080000:3.0:1775765619.722799:0:73045:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765619.722804:0:73045:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765619.722814:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775765619.722824:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114339584:192.168.202.110@tcp:250:kworker.0
00000100:00100000:3.0:1775765619.722833:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744380 x1862022114339712/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765619.722847:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765619.722862:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775765619.722866:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765619.722871:0:73045:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2997
00000100:00080000:3.0:1775765619.722876:0:73045:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765619.722881:0:73045:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765619.722888:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775765619.722905:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746a00 x1862022114339840/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765619.722916:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765619.722921:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775765619.722923:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765619.722928:0:73045:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2997
00000100:00080000:3.0:1775765619.722931:0:73045:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765619.722934:0:73045:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765619.723006:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775765619.723022:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746300 x1862022114339968/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765619.723035:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765619.723038:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775765619.723040:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765619.723044:0:73045:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 2997
00000100:00080000:3.0:1775765619.723056:0:73045:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765619.723062:0:73045:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765619.723069:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775765619.723082:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747b80 x1862022114340096/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765619.723088:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765619.723206:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775765619.723210:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765619.723220:0:73045:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 2997
00000100:00080000:3.0:1775765619.723228:0:73045:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765619.723297:0:73045:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765619.723305:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775765619.723325:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744e00 x1862022114340224/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765619.723358:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765619.723361:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775765619.723363:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765619.723368:0:73045:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 2997
00000100:00080000:3.0:1775765619.723375:0:73045:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765619.723381:0:73045:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765619.723385:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775765619.723397:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746680 x1862022114340352/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765619.723419:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765619.723424:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775765619.723428:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765619.723436:0:73045:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 2997
00000100:00080000:3.0:1775765619.723442:0:73045:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765619.723446:0:73045:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765619.723453:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775765619.723467:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745880 x1862022114340480/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00010000:00010000:2.0:1775765619.978843:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765620.746691:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775765621.002820:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765621.770736:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775765622.026726:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765622.794728:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775765623.050823:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765623.818687:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775765624.074831:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765624.842655:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775765624.842657:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765624.842669:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775765624.842675:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765624.842687:0:73045:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3002
00000100:00080000:3.0:1775765624.842699:0:73045:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765624.842706:0:73045:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775765624.842760:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744a80 x1862022114340608/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765624.842784:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765624.842789:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775765624.842790:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765624.842797:0:73045:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3002
00000100:00080000:3.0:1775765624.842810:0:73045:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765624.842814:0:73045:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765624.842822:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775765624.842833:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745880 x1862022114340736/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765624.842841:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765624.842844:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775765624.842846:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765624.842851:0:73045:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3002
00000100:00080000:3.0:1775765624.842854:0:73045:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765624.842861:0:73045:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765624.842864:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775765624.842872:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746680 x1862022114340864/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765624.842879:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765624.842882:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775765624.842883:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765624.842886:0:73045:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3002
00000100:00080000:3.0:1775765624.842888:0:73045:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765624.842891:0:73045:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765624.842895:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775765624.842901:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744e00 x1862022114340992/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765624.842919:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765624.842921:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775765624.842922:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765624.842925:0:73045:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3002
00000100:00080000:3.0:1775765624.842929:0:73045:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765624.842935:0:73045:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765624.842939:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775765624.842949:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747b80 x1862022114341120/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765624.842955:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765624.842957:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775765624.842959:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765624.842962:0:73045:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3002
00000100:00080000:3.0:1775765624.842965:0:73045:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765624.842977:0:73045:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765624.842983:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775765624.842994:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746300 x1862022114341248/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765624.843013:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765624.843019:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775765624.843023:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765624.843031:0:73045:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3002
00000100:00080000:3.0:1775765624.843036:0:73045:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765624.843040:0:73045:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765624.843046:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775765624.843058:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746a00 x1862022114341376/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765624.843065:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765624.843070:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775765624.843073:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765624.843078:0:73045:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3002
00000100:00080000:3.0:1775765624.843081:0:73045:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765624.843087:0:73045:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765624.843094:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775765624.843103:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744380 x1862022114341504/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775765625.866661:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775765626.122731:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00010000:00010000:2.0:1775765627.146699:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00010000:00010000:2.0:1775765628.170717:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00010000:00010000:1.0:1775765629.194720:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1775765629.962685:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765629.962708:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775765629.962716:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765629.962730:0:73045:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3007
00000100:00080000:3.0:1775765629.962744:0:73045:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765629.962754:0:73045:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775765629.962805:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745500 x1862022114341632/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765629.962862:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765629.962869:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775765629.962873:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765629.962882:0:73045:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3007
00000100:00080000:3.0:1775765629.962890:0:73045:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765629.962899:0:73045:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765629.962916:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775765629.962967:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745c00 x1862022114341760/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765629.962985:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765629.962990:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775765629.962992:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765629.962998:0:73045:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3007
00000100:00080000:3.0:1775765629.963005:0:73045:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765629.963015:0:73045:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765629.963028:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775765629.963048:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745180 x1862022114341888/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765629.963062:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765629.963066:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775765629.963069:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765629.963077:0:73045:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3007
00000100:00080000:3.0:1775765629.963086:0:73045:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765629.963099:0:73045:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765629.963112:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775765629.963131:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747100 x1862022114342016/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765629.963196:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765629.963204:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775765629.963211:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765629.963219:0:73045:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3007
00000100:00080000:3.0:1775765629.963224:0:73045:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765629.963228:0:73045:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765629.963235:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775765629.963251:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744700 x1862022114342144/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765629.963268:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765629.963273:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775765629.963277:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765629.963300:0:73045:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3007
00000100:00080000:3.0:1775765629.963310:0:73045:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765629.963325:0:73045:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765629.963335:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775765629.963364:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745f80 x1862022114342272/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765629.963422:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765629.963429:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775765629.963438:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765629.963450:0:73045:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3007
00000100:00080000:3.0:1775765629.963465:0:73045:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765629.963480:0:73045:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765629.963519:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775765629.963602:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744000 x1862022114342400/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765629.963619:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765629.963631:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775765629.963636:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765629.963645:0:73045:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3007
00000100:00080000:3.0:1775765629.963653:0:73045:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765629.963667:0:73045:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765629.963684:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775765629.963711:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747800 x1862022114342528/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00010000:00010000:1.0:1775765630.218748:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00010000:00010000:2.0:1775765631.242796:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00010000:00010000:2.0:1775765633.290709:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00010000:00010000:2.0:1775765634.314774:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1775765635.082725:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765635.082734:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1775765635.082739:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765635.082750:0:8:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3012
00000100:00080000:1.0:1775765635.082762:0:8:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765635.082770:0:8:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775765635.082838:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fc380 x1862022114342656/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765635.082869:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765635.082873:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775765635.082875:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765635.082879:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3012
00000100:00080000:1.0:1775765635.082883:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765635.082888:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765635.082916:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775765635.082935:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fea00 x1862022114342784/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765635.082957:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765635.082961:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775765635.082963:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765635.082969:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3012
00000100:00080000:1.0:1775765635.082984:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765635.082989:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765635.082995:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775765635.083008:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff100 x1862022114342912/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765635.083019:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765635.083022:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775765635.083024:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765635.083028:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3012
00000100:00080000:1.0:1775765635.083052:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765635.083062:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765635.083068:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775765635.083082:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fe300 x1862022114343040/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765635.083092:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765635.083096:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1775765635.083098:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765635.083102:0:8:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3012
00000100:00080000:1.0:1775765635.083106:0:8:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765635.083125:0:8:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765635.083135:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775765635.083151:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fca80 x1862022114343168/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765635.083181:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765635.083187:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775765635.083190:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765635.083200:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3012
00000100:00080000:1.0:1775765635.083203:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765635.083207:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765635.083210:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775765635.083223:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fed80 x1862022114343296/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765635.083245:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765635.083249:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775765635.083250:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765635.083255:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3012
00000100:00080000:1.0:1775765635.083259:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765635.083262:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765635.083267:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775765635.083280:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff800 x1862022114343424/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765635.083291:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765635.083295:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775765635.083297:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765635.083300:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3012
00000100:00080000:1.0:1775765635.083305:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765635.083314:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765635.083351:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775765635.083369:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ffb80 x1862022114343552/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00010000:00010000:2.0:1775765635.338733:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00010000:00010000:1.0:1775765636.362674:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00010000:00010000:1.0:1775765637.386794:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00010000:00010000:2.0:1775765638.410769:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765640.202908:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114343680:192.168.202.110@tcp:250:kworker.0
00000800:00000100:2.0:1775765640.203001:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775765640.203053:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765640/real 1775765640]  req@ffff9f14a04d0380 x1862022114343680/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765665 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775765640.203086:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0380 x1862022114343680/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765665 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765640.203106:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775765640.203118:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775765640.203131:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114343680:192.168.202.110@tcp:250:kworker.0
00000100:00100000:2.0:1775765640.203150:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114343808:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775765640.203193:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775765640.203209:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765640/real 1775765640]  req@ffff9f14a04d1f80 x1862022114343808/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765665 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775765640.203221:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1f80 x1862022114343808/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765665 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765640.203227:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775765640.203230:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775765640.203232:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114343808:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775765640.203253:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114343936:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775765640.203269:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775765640.203277:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765640/real 1775765640]  req@ffff9f14a04d3b80 x1862022114343936/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765665 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775765640.203286:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3b80 x1862022114343936/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765665 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765640.203291:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775765640.203293:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775765640.203295:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114343936:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775765640.203299:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114344064:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775765640.203310:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775765640.203316:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765640/real 1775765640]  req@ffff9f14a04d3100 x1862022114344064/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765665 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775765640.203341:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3100 x1862022114344064/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765665 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765640.203348:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775765640.203350:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775765640.203353:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114344064:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775765640.203357:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114344192:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775765640.203369:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775765640.203376:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765640/real 1775765640]  req@ffff9f14a04d0a80 x1862022114344192/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765665 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775765640.203385:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0a80 x1862022114344192/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765665 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765640.203394:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775765640.203418:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775765640.203423:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114344192:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775765640.203431:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114344320:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775765640.203459:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775765640.203473:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765640/real 1775765640]  req@ffff9f14a04d2a00 x1862022114344320/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765665 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775765640.203488:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d2a00 x1862022114344320/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765665 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765640.203495:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775765640.203497:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775765640.203500:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114344320:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775765640.203505:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114344448:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775765640.203524:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775765640.203532:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765640/real 1775765640]  req@ffff9f14a04d0e00 x1862022114344448/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765665 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775765640.203541:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0e00 x1862022114344448/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765665 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765640.203547:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775765640.203550:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775765640.203582:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114344448:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775765640.203588:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114344576:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775765640.203605:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775765640.203614:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765640/real 1775765640]  req@ffff9f14a04d1180 x1862022114344576/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765665 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775765640.203623:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1180 x1862022114344576/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765665 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765640.203630:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775765640.203632:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775765640.203635:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114344576:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775765640.458680:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765641.226673:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775765641.482818:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765642.250643:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775765642.506690:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765643.274708:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775765643.530840:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765644.298633:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775765644.554709:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765645.322639:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:1.0:1775765645.322710:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765645.322720:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1775765645.322725:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765645.322738:0:8:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3022
00000100:00080000:1.0:1775765645.322753:0:8:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765645.322762:0:8:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775765645.322814:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdc00 x1862022114344704/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765645.322852:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765645.322856:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775765645.322858:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765645.322864:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3022
00000100:00080000:1.0:1775765645.322870:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765645.322875:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765645.322887:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775765645.322888:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114344704:192.168.202.110@tcp:250:kworker.0
00000100:00100000:1.0:1775765645.322917:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fce00 x1862022114344832/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765645.322933:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765645.322939:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775765645.322941:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765645.322946:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3022
00000100:00080000:1.0:1775765645.322952:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765645.322957:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000800:00000100:2.0:1775765645.322962:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000008:00100000:1.0:1775765645.322972:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775765645.322993:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fd880 x1862022114344960/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765645.323006:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:2.0:1775765645.323012:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765645/real 1775765645]  req@ffff9f14ab4fdc00 x1862022114344704/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765670 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765645.323014:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775765645.323022:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765645.323038:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3022
00000100:00100000:2.0:1775765645.323043:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdc00 x1862022114344704/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765670 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765645.323044:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765645.323061:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00080000:2.0:1775765645.323062:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775765645.323074:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000008:00100000:1.0:1775765645.323077:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775765645.323080:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114344704:192.168.202.110@tcp:250:kworker.0
00000100:00100000:1.0:1775765645.323105:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdf80 x1862022114345088/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775765645.323109:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114344832:192.168.202.110@tcp:38:kworker.0
00000100:00080000:1.0:1775765645.323130:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765645.323134:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1775765645.323140:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000800:00000100:2.0:1775765645.323140:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:1.0:1775765645.323151:0:8:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3022
00000100:00100000:2.0:1775765645.323156:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765645/real 1775765645]  req@ffff9f14ab4fce00 x1862022114344832/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765670 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765645.323157:0:8:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765645.323172:0:8:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775765645.323179:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fce00 x1862022114344832/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765670 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:1.0:1775765645.323184:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00080000:2.0:1775765645.323186:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775765645.323194:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775765645.323199:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114344832:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765645.323211:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdc00 x1862022114345216/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775765645.323211:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114344960:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775765645.323236:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:1.0:1775765645.323237:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765645.323243:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00100000:2.0:1775765645.323246:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765645/real 1775765645]  req@ffff9f14ab4fd880 x1862022114344960/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765670 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765645.323247:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765645.323264:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3022
00000100:00100000:2.0:1775765645.323270:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fd880 x1862022114344960/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765670 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765645.323274:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765645.323284:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765645.323290:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00080000:2.0:1775765645.323295:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775765645.323304:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114344960:192.168.202.110@tcp:38:kworker.0
00000008:00100000:1.0:1775765645.323306:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775765645.323316:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114345088:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775765645.323325:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ffb80 x1862022114345344/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:2.0:1775765645.323337:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775765645.323346:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765645/real 1775765645]  req@ffff9f14ab4fdf80 x1862022114345088/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765670 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775765645.323370:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdf80 x1862022114345088/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765670 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765645.323376:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775765645.323384:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775765645.323394:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114345088:192.168.202.110@tcp:8:kworker.0
00000100:00080000:1.0:1775765645.323399:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:2.0:1775765645.323404:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114345216:192.168.202.110@tcp:8:kworker.0
00000100:00080000:1.0:1775765645.323415:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775765645.323419:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765645.323428:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3022
00000800:00000100:2.0:1775765645.323432:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:1.0:1775765645.323433:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765645.323441:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775765645.323444:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765645/real 1775765645]  req@ffff9f14ab4fdc00 x1862022114345216/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765670 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:1.0:1775765645.323458:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775765645.323462:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdc00 x1862022114345216/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765670 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765645.323470:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775765645.323473:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775765645.323474:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114345216:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775765645.323484:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ffb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114345344:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765645.323491:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff800 x1862022114345472/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:2.0:1775765645.323502:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775765645.323515:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765645/real 1775765645]  req@ffff9f14ab4ffb80 x1862022114345344/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765670 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765645.323516:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765645.323535:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775765645.323541:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:2.0:1775765645.323544:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ffb80 x1862022114345344/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765670 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765645.323578:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775765645.323581:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775765645.323583:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ffb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114345344:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775765645.323593:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114345472:192.168.202.110@tcp:38:kworker.0
00000100:00080000:1.0:1775765645.323604:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3022
00000800:00000100:2.0:1775765645.323612:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:1.0:1775765645.323615:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:2.0:1775765645.323622:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765645/real 1775765645]  req@ffff9f14ab4ff800 x1862022114345472/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765670 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765645.323624:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765645.323643:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775765645.323647:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff800 x1862022114345472/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765670 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765645.323661:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775765645.323665:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775765645.323668:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114345472:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765645.323673:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fed80 x1862022114345600/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775765645.323705:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114345600:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775765645.323722:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775765645.323734:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765645/real 1775765645]  req@ffff9f14ab4fed80 x1862022114345600/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765670 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775765645.323742:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fed80 x1862022114345600/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765670 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765645.323747:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775765645.323749:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775765645.323751:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114345600:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775765645.578686:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765646.346664:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:2.0:1775765647.370734:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775765647.626767:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765648.394630:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775765648.650854:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765649.418633:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775765649.674887:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765650.442668:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:1.0:1775765650.442685:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765650.442692:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1775765650.442695:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765650.442703:0:73045:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3027
00000100:00080000:1.0:1775765650.442713:0:73045:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765650.442718:0:73045:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775765650.442753:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fca80 x1862022114345728/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765650.442787:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765650.442804:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775765650.442806:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765650.442810:0:73045:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3027
00000100:00080000:1.0:1775765650.442813:0:73045:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765650.442816:0:73045:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775765650.442821:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114345728:192.168.202.110@tcp:250:kworker.0
00000008:00100000:1.0:1775765650.442847:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775765650.442862:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fe300 x1862022114345856/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765650.442885:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765650.442888:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775765650.442892:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000800:00000100:2.0:1775765650.442892:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:1.0:1775765650.442900:0:73045:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3027
00000100:00080000:1.0:1775765650.442924:0:73045:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765650.442927:0:73045:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765650.442931:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775765650.442946:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765650/real 1775765650]  req@ffff9f14ab4fca80 x1862022114345728/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765675 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765650.442947:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff100 x1862022114345984/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775765650.442966:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fca80 x1862022114345728/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765675 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765650.442984:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775765650.442992:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775765650.442999:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114345728:192.168.202.110@tcp:250:kworker.0
00000100:00080000:1.0:1775765650.443001:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:2.0:1775765650.443017:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114345856:192.168.202.110@tcp:38:kworker.0
00000100:00080000:1.0:1775765650.443020:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775765650.443023:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000800:00000100:2.0:1775765650.443037:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:1.0:1775765650.443039:0:73045:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3027
00000100:00080000:1.0:1775765650.443044:0:73045:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765650.443049:0:73045:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775765650.443052:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765650/real 1775765650]  req@ffff9f14ab4fe300 x1862022114345856/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765675 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:1.0:1775765650.443056:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775765650.443065:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fe300 x1862022114345856/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765675 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765650.443072:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775765650.443075:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775765650.443078:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114345856:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765650.443083:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fea00 x1862022114346112/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775765650.443085:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114345984:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775765650.443101:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775765650.443111:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765650/real 1775765650]  req@ffff9f14ab4ff100 x1862022114345984/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765675 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775765650.443119:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff100 x1862022114345984/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765675 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765650.443131:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765650.443131:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775765650.443138:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00080000:1.0:1775765650.443141:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00100000:2.0:1775765650.443143:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114345984:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775765650.443152:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114346112:192.168.202.110@tcp:8:kworker.0
00000100:00080000:1.0:1775765650.443157:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765650.443163:0:73045:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3027
00000800:00000100:2.0:1775765650.443167:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775765650.443173:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765650/real 1775765650]  req@ffff9f14ab4fea00 x1862022114346112/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765675 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775765650.443181:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fea00 x1862022114346112/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765675 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765650.443184:0:73045:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765650.443188:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765650.443194:0:73045:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00080000:2.0:1775765650.443194:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775765650.443197:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114346112:192.168.202.110@tcp:8:kworker.0
00000008:00100000:1.0:1775765650.443218:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775765650.443247:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fc380 x1862022114346240/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765650.443270:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765650.443273:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775765650.443274:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765650.443288:0:73045:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3027
00000100:00100000:2.0:1775765650.443294:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114346240:192.168.202.110@tcp:8:kworker.0
00000100:00080000:1.0:1775765650.443298:0:73045:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765650.443357:0:73045:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000800:00000100:2.0:1775765650.443362:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000008:00100000:1.0:1775765650.443363:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775765650.443384:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765650/real 1775765650]  req@ffff9f14ab4fc380 x1862022114346240/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765675 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775765650.443394:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fc380 x1862022114346240/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765675 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765650.443401:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fea00 x1862022114346368/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775765650.443403:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775765650.443412:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775765650.443418:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114346240:192.168.202.110@tcp:8:kworker.0
00000100:00080000:1.0:1775765650.443427:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:2.0:1775765650.443430:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114346368:192.168.202.110@tcp:38:kworker.0
00000100:00080000:1.0:1775765650.443433:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775765650.443437:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765650.443441:0:73045:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3027
00000800:00000100:2.0:1775765650.443449:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:1.0:1775765650.443455:0:73045:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765650.443461:0:73045:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775765650.443462:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765650/real 1775765650]  req@ffff9f14ab4fea00 x1862022114346368/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765675 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:1.0:1775765650.443467:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775765650.443473:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fea00 x1862022114346368/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765675 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765650.443484:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775765650.443487:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775765650.443493:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114346368:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765650.443505:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff100 x1862022114346496/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765650.443532:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:2.0:1775765650.443555:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114346496:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775765650.443598:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775765650.443623:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765650/real 1775765650]  req@ffff9f14ab4ff100 x1862022114346496/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765675 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775765650.443633:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff100 x1862022114346496/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765675 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765650.443638:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775765650.443641:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775765650.443642:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114346496:192.168.202.110@tcp:38:kworker.0
00000100:00080000:1.0:1775765650.443670:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775765650.443672:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765650.443677:0:73045:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3027
00000100:00080000:1.0:1775765650.443681:0:73045:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765650.443684:0:73045:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765650.443689:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775765650.443699:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fe300 x1862022114346624/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775765650.443738:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114346624:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775765650.443763:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775765650.443772:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765650/real 1775765650]  req@ffff9f14ab4fe300 x1862022114346624/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765675 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775765650.443780:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fe300 x1862022114346624/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765675 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765650.443786:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775765650.443788:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775765650.443790:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114346624:192.168.202.110@tcp:8:kworker.0
00010000:00010000:1.0:1775765650.698730:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765651.466643:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775765651.722698:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765652.490641:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775765652.746708:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765653.514663:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:2.0:1775765654.538686:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775765654.794870:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765655.562657:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:2.0:1775765655.562868:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114346752:192.168.202.110@tcp:250:kworker.0
00000800:00000100:2.0:1775765655.562955:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775765655.563039:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765655/real 1775765655]  req@ffff9f14a04d3480 x1862022114346752/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765680 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775765655.563081:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3480 x1862022114346752/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765680 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765655.563095:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775765655.563109:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775765655.563119:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114346752:192.168.202.110@tcp:250:kworker.0
00000100:00100000:2.0:1775765655.563181:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114346880:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775765655.563223:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775765655.563243:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765655/real 1775765655]  req@ffff9f14a04d0000 x1862022114346880/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765680 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775765655.563306:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0000 x1862022114346880/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765680 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765655.563317:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775765655.563321:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775765655.563325:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114346880:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775765655.563334:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114347008:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775765655.563363:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775765655.563380:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765655/real 1775765655]  req@ffff9f14a04d1500 x1862022114347008/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765680 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775765655.563397:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1500 x1862022114347008/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765680 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765655.563407:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775765655.563412:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775765655.563416:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114347008:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775765655.563425:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114347136:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775765655.563455:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775765655.563476:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765655/real 1775765655]  req@ffff9f14a04d2300 x1862022114347136/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765680 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775765655.563512:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d2300 x1862022114347136/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765680 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765655.563523:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775765655.563527:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775765655.563531:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114347136:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775765655.563539:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114347264:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775765655.563602:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775765655.563630:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765655/real 1775765655]  req@ffff9f14a04d3800 x1862022114347264/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765680 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775765655.563655:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3800 x1862022114347264/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765680 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765655.563664:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775765655.563668:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775765655.563675:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114347264:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775765655.563685:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114347392:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775765655.563714:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775765655.563728:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765655/real 1775765655]  req@ffff9f14a04d1180 x1862022114347392/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765680 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775765655.563740:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1180 x1862022114347392/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765680 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765655.563770:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775765655.563773:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775765655.563775:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114347392:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775765655.563799:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114347520:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775765655.563817:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775765655.563826:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765655/real 1775765655]  req@ffff9f14a04d0e00 x1862022114347520/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765680 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775765655.563836:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0e00 x1862022114347520/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765680 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765655.563843:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775765655.563846:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775765655.563848:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114347520:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775765655.563853:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114347648:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775765655.563866:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775765655.563879:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765655/real 1775765655]  req@ffff9f14a04d2a00 x1862022114347648/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765680 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775765655.563896:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d2a00 x1862022114347648/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765680 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765655.563907:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775765655.563912:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775765655.563918:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114347648:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775765655.818786:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765656.586637:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775765656.842710:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765657.610653:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775765657.866900:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765658.634632:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775765658.890758:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765659.658706:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775765659.914739:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765660.682668:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:2.0:1775765660.682801:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114347776:192.168.202.110@tcp:250:kworker.0
00000800:00000100:2.0:1775765660.682903:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775765660.682956:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765660/real 1775765660]  req@ffff9f14a04d0a80 x1862022114347776/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765685 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775765660.683001:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0a80 x1862022114347776/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765685 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765660.683046:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775765660.683059:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775765660.683072:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114347776:192.168.202.110@tcp:250:kworker.0
00000100:00100000:2.0:1775765660.683118:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114347904:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775765660.683144:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775765660.683157:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765660/real 1775765660]  req@ffff9f14a04d3100 x1862022114347904/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765685 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775765660.683189:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3100 x1862022114347904/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765685 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765660.683228:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775765660.683235:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775765660.683242:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114347904:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775765660.683248:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114348032:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775765660.683271:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775765660.683288:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765660/real 1775765660]  req@ffff9f14a04d3b80 x1862022114348032/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765685 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775765660.683329:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3b80 x1862022114348032/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765685 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765660.683343:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775765660.683353:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775765660.683361:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114348032:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775765660.683367:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114348160:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775765660.683398:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775765660.683411:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765660/real 1775765660]  req@ffff9f14a04d1f80 x1862022114348160/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765685 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775765660.683423:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1f80 x1862022114348160/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765685 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765660.683430:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775765660.683432:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775765660.683436:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114348160:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775765660.683442:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114348288:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775765660.683455:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775765660.683463:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765660/real 1775765660]  req@ffff9f14a04d0380 x1862022114348288/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765685 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775765660.683473:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0380 x1862022114348288/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765685 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765660.683489:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775765660.683508:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775765660.683510:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114348288:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775765660.683525:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114348416:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775765660.683538:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775765660.683545:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765660/real 1775765660]  req@ffff9f14a04d0a80 x1862022114348416/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765685 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775765660.683554:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0a80 x1862022114348416/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765685 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765660.683559:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775765660.683561:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775765660.683563:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114348416:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775765660.683567:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114348544:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775765660.683578:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775765660.683607:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765660/real 1775765660]  req@ffff9f14a04d2a00 x1862022114348544/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765685 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775765660.683617:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d2a00 x1862022114348544/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765685 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765660.683623:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775765660.683625:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775765660.683628:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114348544:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775765660.683632:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114348672:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775765660.683647:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775765660.683654:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765660/real 1775765660]  req@ffff9f14a04d0e00 x1862022114348672/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765685 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775765660.683662:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0e00 x1862022114348672/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765685 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765660.683668:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775765660.683670:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775765660.683672:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114348672:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775765661.706633:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775765661.962798:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765662.730743:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775765662.986818:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765663.754634:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775765664.010678:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765664.778750:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775765665.034817:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765665.802695:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:2.0:1775765665.802902:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114348800:192.168.202.110@tcp:250:kworker.0
00000100:00100000:2.0:1775765665.803148:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114348928:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775765665.803249:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114349056:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775765665.803307:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114349184:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775765665.803350:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114349312:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775765665.803397:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114349440:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775765665.803417:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114349568:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775765665.803443:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114349696:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775765666.058811:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765666.826658:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775765667.082767:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765667.850693:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:2.0:1775765668.874675:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000400:00000100:3.0:1775765668.874870:0:2390:0:(lib-socket.c:454:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.202.110:988
00000400:00000100:3.0:1775765668.874927:0:2390:0:(acceptor.c:111:lnet_connect_console_error()) Connection to 192.168.202.110@tcp at host 192.168.202.110 was unreachable: the network or that node may be down, or Lustre may be misconfigured.
00000800:00000100:3.0:1775765668.874946:0:2390:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:3.0:1775765668.874986:0:2390:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:3.0:1775765668.874996:0:2390:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:3.0:1775765668.875009:0:2390:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:3.0:1775765668.875019:0:2390:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:3.0:1775765668.875033:0:2390:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000100:00100000:2.0:1775765668.875047:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765665/real 1775765668]  req@ffff9f14a04d3800 x1862022114348928/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765690 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000800:00000100:3.0:1775765668.875047:0:2390:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:3.0:1775765668.875072:0:2390:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000100:00100000:2.0:1775765668.875104:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3800 x1862022114348928/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765690 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765668.875118:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775765668.875123:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775765668.875154:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114348928:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775765668.875163:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765665/real 1775765668]  req@ffff9f14a04d2300 x1862022114349056/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765690 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775765668.875179:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d2300 x1862022114349056/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765690 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765668.875189:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775765668.875195:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775765668.875201:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114349056:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775765668.875212:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765665/real 1775765668]  req@ffff9f14a04d1500 x1862022114349184/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765690 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775765668.875233:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1500 x1862022114349184/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765690 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765668.875270:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775765668.875283:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775765668.875289:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114349184:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775765668.875301:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765665/real 1775765668]  req@ffff9f14a04d0000 x1862022114349312/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765690 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775765668.875324:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0000 x1862022114349312/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765690 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765668.875336:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775765668.875341:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775765668.875346:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114349312:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775765668.875356:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765665/real 1775765668]  req@ffff9f14a04d1180 x1862022114348800/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765690 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775765668.875376:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1180 x1862022114348800/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765690 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765668.875388:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775765668.875394:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775765668.875399:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114348800:192.168.202.110@tcp:250:kworker.0
00000100:00100000:2.0:1775765668.875447:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765665/real 1775765668]  req@ffff9f14a04d3480 x1862022114349440/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765690 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775765668.875469:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3480 x1862022114349440/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765690 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765668.875484:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775765668.875489:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775765668.875495:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114349440:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775765668.875505:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765665/real 1775765668]  req@ffff9f14a04d0e00 x1862022114349568/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765690 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775765668.875527:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0e00 x1862022114349568/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765690 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765668.875540:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775765668.875546:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775765668.875551:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114349568:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775765668.875600:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765665/real 1775765668]  req@ffff9f14a04d2a00 x1862022114349696/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765690 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775765668.875623:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d2a00 x1862022114349696/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765690 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765668.875636:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775765668.875641:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775765668.875645:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114349696:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775765669.130934:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765669.898679:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775765670.154842:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765670.922797:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:1.0:1775765670.922918:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114349824:192.168.202.110@tcp:250:kworker.0
00000800:00000100:1.0:1775765670.923010:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765670.923081:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765670/real 1775765670]  req@ffff9f14a04d0a80 x1862022114349824/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765695 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765670.923171:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0a80 x1862022114349824/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765695 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765670.923219:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765670.923237:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765670.923257:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114349824:192.168.202.110@tcp:250:kworker.0
00000100:00100000:1.0:1775765670.923311:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114349952:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765670.923387:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765670.923423:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765670/real 1775765670]  req@ffff9f14a04d0380 x1862022114349952/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765695 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765670.923460:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0380 x1862022114349952/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765695 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765670.923497:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765670.923514:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765670.923532:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114349952:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765670.923676:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114350080:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765670.923753:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765670.923790:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765670/real 1775765670]  req@ffff9f14a04d1f80 x1862022114350080/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765695 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765670.923847:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1f80 x1862022114350080/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765695 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765670.923876:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765670.923891:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765670.923899:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114350080:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765670.923952:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114350208:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765670.924003:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765670.924025:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765670/real 1775765670]  req@ffff9f14a04d3b80 x1862022114350208/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765695 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765670.924063:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3b80 x1862022114350208/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765695 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765670.924081:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765670.924087:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765670.924095:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114350208:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775765670.924132:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114350336:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765670.924175:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765670.924197:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765670/real 1775765670]  req@ffff9f14a04d3100 x1862022114350336/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765695 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765670.924222:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3100 x1862022114350336/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765695 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765670.924239:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765670.924246:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765670.924253:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114350336:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775765670.924265:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114350464:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765670.924305:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765670.924326:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765670/real 1775765670]  req@ffff9f14a04d0a80 x1862022114350464/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765695 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765670.924351:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0a80 x1862022114350464/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765695 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765670.924380:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765670.924387:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765670.924393:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114350464:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765670.924405:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114350592:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765670.924442:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765670.924464:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765670/real 1775765670]  req@ffff9f14a04d2a00 x1862022114350592/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765695 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765670.924489:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d2a00 x1862022114350592/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765695 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765670.924505:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765670.924511:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765670.924518:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114350592:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765670.924595:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114350720:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765670.924639:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765670.924662:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765670/real 1775765670]  req@ffff9f14a04d0e00 x1862022114350720/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765695 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765670.924687:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0e00 x1862022114350720/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765695 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765670.924703:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765670.924708:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765670.924715:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114350720:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775765671.178871:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765671.946670:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775765672.202809:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765672.970635:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775765673.226720:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765673.994628:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775765674.250681:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765675.018659:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:2.0:1775765676.042606:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:1.0:1775765676.042791:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114350848:192.168.202.110@tcp:250:kworker.0
00000800:00000100:1.0:1775765676.042857:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765676.042895:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765676/real 1775765676]  req@ffff9f14a04d3480 x1862022114350848/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765701 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765676.042930:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3480 x1862022114350848/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765701 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765676.042946:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765676.042952:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765676.042956:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114350848:192.168.202.110@tcp:250:kworker.0
00000100:00100000:1.0:1775765676.042966:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114350976:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765676.042991:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765676.043003:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765676/real 1775765676]  req@ffff9f14a04d1180 x1862022114350976/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765701 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765676.043019:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1180 x1862022114350976/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765701 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765676.043043:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765676.043051:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765676.043054:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114350976:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765676.043086:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114351104:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765676.043107:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765676.043119:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765676/real 1775765676]  req@ffff9f14a04d0000 x1862022114351104/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765701 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765676.043127:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0000 x1862022114351104/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765701 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765676.043133:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765676.043135:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765676.043137:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114351104:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765676.043141:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114351232:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765676.043151:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765676.043156:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765676/real 1775765676]  req@ffff9f14a04d1500 x1862022114351232/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765701 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765676.043164:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1500 x1862022114351232/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765701 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765676.043169:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765676.043170:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765676.043172:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114351232:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775765676.043175:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114351360:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765676.043184:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765676.043189:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765676/real 1775765676]  req@ffff9f14a04d2300 x1862022114351360/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765701 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765676.043196:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d2300 x1862022114351360/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765701 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765676.043201:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765676.043203:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765676.043206:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114351360:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775765676.043209:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114351488:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765676.043218:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765676.043223:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765676/real 1775765676]  req@ffff9f14a04d3800 x1862022114351488/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765701 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765676.043229:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3800 x1862022114351488/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765701 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765676.043234:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765676.043236:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765676.043238:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114351488:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765676.043241:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114351616:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765676.043250:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765676.043254:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765676/real 1775765676]  req@ffff9f14a04d0e00 x1862022114351616/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765701 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765676.043261:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0e00 x1862022114351616/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765701 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765676.043265:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765676.043266:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765676.043269:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114351616:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765676.043282:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114351744:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765676.043292:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765676.043297:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765676/real 1775765676]  req@ffff9f14a04d2a00 x1862022114351744/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765701 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765676.043304:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d2a00 x1862022114351744/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765701 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765676.043309:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765676.043311:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765676.043312:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114351744:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775765676.298727:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765677.066647:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775765677.322738:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765678.090642:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775765678.346763:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765679.114621:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775765679.370718:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765680.138679:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775765680.394701:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765681.162630:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:1.0:1775765681.162979:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114351872:192.168.202.110@tcp:250:kworker.0
00000800:00000100:1.0:1775765681.163053:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765681.163089:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765681/real 1775765681]  req@ffff9f14a04d0a80 x1862022114351872/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765706 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765681.163109:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0a80 x1862022114351872/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765706 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765681.163117:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765681.163122:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765681.163125:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114351872:192.168.202.110@tcp:250:kworker.0
00000100:00100000:1.0:1775765681.163145:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114352000:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765681.163165:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765681.163183:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765681/real 1775765681]  req@ffff9f14a04d3100 x1862022114352000/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765706 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765681.163202:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3100 x1862022114352000/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765706 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765681.163212:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765681.163219:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765681.163224:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114352000:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765681.163241:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114352128:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765681.163269:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765681.163280:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765681/real 1775765681]  req@ffff9f14a04d3b80 x1862022114352128/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765706 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765681.163295:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3b80 x1862022114352128/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765706 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765681.163301:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765681.163303:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765681.163305:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114352128:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765681.163316:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114352256:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765681.163332:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765681.163338:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765681/real 1775765681]  req@ffff9f14a04d1f80 x1862022114352256/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765706 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765681.163348:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1f80 x1862022114352256/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765706 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765681.163372:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765681.163375:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765681.163377:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114352256:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775765681.163439:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114352384:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765681.163479:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765681.163495:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765681/real 1775765681]  req@ffff9f14a04d0380 x1862022114352384/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765706 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765681.163503:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0380 x1862022114352384/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765706 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765681.163508:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765681.163513:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765681.163516:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114352384:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775765681.163702:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114352512:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765681.163729:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765681.163743:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765681/real 1775765681]  req@ffff9f14a04d0380 x1862022114352512/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765706 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765681.163755:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0380 x1862022114352512/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765706 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765681.163762:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765681.163769:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765681.163772:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114352512:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765681.163787:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114352640:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765681.163804:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765681.163815:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765681/real 1775765681]  req@ffff9f14a04d1f80 x1862022114352640/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765706 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765681.163823:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1f80 x1862022114352640/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765706 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765681.163832:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765681.163835:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765681.163840:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114352640:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765681.163874:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114352768:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765681.163894:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765681.163902:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765681/real 1775765681]  req@ffff9f14a04d3b80 x1862022114352768/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765706 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765681.163909:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3b80 x1862022114352768/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765706 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765681.163915:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765681.163917:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765681.163919:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114352768:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775765681.418786:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765682.186657:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:2.0:1775765683.210626:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775765683.466662:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765684.234629:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775765684.490692:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765685.258747:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775765685.514774:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765686.282708:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:1.0:1775765686.283016:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114352896:192.168.202.110@tcp:250:kworker.0
00000800:00000100:1.0:1775765686.283156:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765686.283230:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765686/real 1775765686]  req@ffff9f14a04d3100 x1862022114352896/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765711 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765686.283271:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3100 x1862022114352896/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765711 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765686.283289:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765686.283296:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765686.283303:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114352896:192.168.202.110@tcp:250:kworker.0
00000100:00100000:1.0:1775765686.283320:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114353024:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765686.283355:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765686.283371:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765686/real 1775765686]  req@ffff9f14a04d0a80 x1862022114353024/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765711 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765686.283387:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0a80 x1862022114353024/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765711 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765686.283397:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765686.283401:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765686.283405:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114353024:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765686.283413:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114353152:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765686.283431:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765686.283442:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765686/real 1775765686]  req@ffff9f14a04d2a00 x1862022114353152/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765711 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765686.283456:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d2a00 x1862022114353152/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765711 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765686.283474:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765686.283478:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765686.283482:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114353152:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765686.283519:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114353280:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765686.283539:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765686.283549:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765686/real 1775765686]  req@ffff9f14a04d0e00 x1862022114353280/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765711 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765686.283610:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0e00 x1862022114353280/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765711 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765686.283621:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765686.283625:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765686.283629:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114353280:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775765686.283637:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114353408:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765686.283661:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765686.283674:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765686/real 1775765686]  req@ffff9f14a04d3800 x1862022114353408/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765711 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765686.283688:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3800 x1862022114353408/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765711 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765686.283698:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765686.283701:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765686.283706:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114353408:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775765686.283713:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114353536:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765686.283732:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765686.283742:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765686/real 1775765686]  req@ffff9f14a04d2300 x1862022114353536/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765711 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765686.283755:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d2300 x1862022114353536/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765711 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765686.283765:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765686.283769:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765686.283772:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114353536:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765686.283779:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114353664:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765686.283797:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765686.283807:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765686/real 1775765686]  req@ffff9f14a04d1500 x1862022114353664/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765711 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765686.283821:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1500 x1862022114353664/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765711 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765686.283830:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765686.283834:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765686.283837:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114353664:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765686.283844:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114353792:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765686.283861:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765686.283872:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765686/real 1775765686]  req@ffff9f14a04d0000 x1862022114353792/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765711 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765686.283886:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0000 x1862022114353792/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765711 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765686.283896:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765686.283900:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765686.283903:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114353792:192.168.202.110@tcp:8:kworker.0
00010000:00010000:3.0:1775765686.538731:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765687.306630:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775765687.562720:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765688.330653:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775765688.586746:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765689.354684:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:2.0:1775765690.378634:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775765690.634816:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765691.402657:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:3.0:1775765691.402855:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114353920:192.168.202.110@tcp:250:kworker.0
00000800:00000100:3.0:1775765691.402958:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775765691.403237:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765691/real 1775765691]  req@ffff9f14a04d1180 x1862022114353920/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765716 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775765691.403256:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1180 x1862022114353920/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765716 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775765691.403265:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775765691.403270:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775765691.403274:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114353920:192.168.202.110@tcp:250:kworker.0
00000100:00100000:3.0:1775765691.403299:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114354048:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775765691.403324:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775765691.403336:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765691/real 1775765691]  req@ffff9f14a04d3480 x1862022114354048/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765716 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775765691.403345:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3480 x1862022114354048/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765716 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775765691.403356:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775765691.403358:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775765691.403360:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114354048:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775765691.403364:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114354176:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775765691.403375:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775765691.403457:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765691/real 1775765691]  req@ffff9f14a04d0000 x1862022114354176/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765716 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775765691.403466:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0000 x1862022114354176/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765716 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775765691.403472:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775765691.403475:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775765691.403477:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114354176:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775765691.403480:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114354304:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775765691.403496:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775765691.403502:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765691/real 1775765691]  req@ffff9f14a04d1500 x1862022114354304/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765716 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775765691.403510:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1500 x1862022114354304/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765716 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775765691.403515:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775765691.403517:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775765691.403520:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114354304:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775765691.403524:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114354432:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775765691.403534:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775765691.403540:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765691/real 1775765691]  req@ffff9f14a04d2300 x1862022114354432/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765716 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775765691.403547:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d2300 x1862022114354432/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765716 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775765691.403552:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775765691.403554:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775765691.403579:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114354432:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775765691.403586:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114354560:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775765691.403599:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775765691.403606:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765691/real 1775765691]  req@ffff9f14a04d3800 x1862022114354560/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765716 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775765691.403614:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3800 x1862022114354560/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765716 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775765691.403620:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775765691.403622:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775765691.403624:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114354560:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775765691.403628:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114354688:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775765691.403639:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775765691.403648:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765691/real 1775765691]  req@ffff9f14a04d0e00 x1862022114354688/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765716 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775765691.403660:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0e00 x1862022114354688/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765716 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775765691.403666:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775765691.403669:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775765691.403671:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114354688:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775765691.403674:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114354816:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775765691.403684:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775765691.403690:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765691/real 1775765691]  req@ffff9f14a04d2a00 x1862022114354816/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765716 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775765691.403697:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d2a00 x1862022114354816/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765716 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775765691.403702:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775765691.403704:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775765691.403705:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114354816:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775765691.658758:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765692.426627:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775765692.682751:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765693.450617:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775765693.706678:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765694.474627:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775765694.730771:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765695.498621:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775765695.754696:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765696.522674:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:3.0:1775765696.522826:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114354944:192.168.202.110@tcp:250:kworker.0
00000800:00000100:3.0:1775765696.522927:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775765696.522983:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765696/real 1775765696]  req@ffff9f14a04d0a80 x1862022114354944/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765721 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775765696.523024:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0a80 x1862022114354944/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765721 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775765696.523042:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775765696.523051:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775765696.523060:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114354944:192.168.202.110@tcp:250:kworker.0
00000100:00100000:3.0:1775765696.523092:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114355072:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775765696.523130:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775765696.523146:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765696/real 1775765696]  req@ffff9f14a04d3100 x1862022114355072/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765721 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775765696.523161:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3100 x1862022114355072/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765721 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775765696.523171:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775765696.523175:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775765696.523178:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114355072:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775765696.523184:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114355200:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775765696.523202:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775765696.523211:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765696/real 1775765696]  req@ffff9f14a04d3b80 x1862022114355200/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765721 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775765696.523223:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3b80 x1862022114355200/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765721 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775765696.523241:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775765696.523244:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775765696.523248:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114355200:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775765696.523256:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114355328:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775765696.523274:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775765696.523283:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765696/real 1775765696]  req@ffff9f14a04d1f80 x1862022114355328/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765721 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775765696.523295:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1f80 x1862022114355328/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765721 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775765696.523304:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775765696.523307:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775765696.523311:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114355328:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775765696.523316:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114355456:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775765696.523332:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775765696.523340:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765696/real 1775765696]  req@ffff9f14a04d0380 x1862022114355456/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765721 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775765696.523351:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0380 x1862022114355456/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765721 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775765696.523360:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775765696.523363:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775765696.523367:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114355456:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775765696.523373:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114355584:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775765696.523388:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775765696.523396:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765696/real 1775765696]  req@ffff9f14a04d2a00 x1862022114355584/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765721 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775765696.523410:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d2a00 x1862022114355584/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765721 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775765696.523419:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775765696.523421:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775765696.523424:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114355584:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775765696.523430:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114355712:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775765696.523446:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775765696.523455:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765696/real 1775765696]  req@ffff9f14a04d0e00 x1862022114355712/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765721 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775765696.523467:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0e00 x1862022114355712/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765721 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775765696.523475:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775765696.523479:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775765696.523482:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114355712:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775765696.523503:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114355840:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775765696.523519:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775765696.523527:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765696/real 1775765696]  req@ffff9f14a04d3800 x1862022114355840/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765721 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775765696.523540:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3800 x1862022114355840/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765721 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775765696.523596:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775765696.523600:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775765696.523603:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114355840:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775765697.546625:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775765697.802731:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765698.570624:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775765698.826715:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765699.594612:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775765699.850771:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765700.618666:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775765700.874684:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765701.642692:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:1.0:1775765701.642714:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765701.642722:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1775765701.642725:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765701.642736:0:73045:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3079
00000100:00080000:1.0:1775765701.642746:0:73045:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765701.642751:0:73045:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775765701.642783:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fca80 x1862022114355968/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765701.642822:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765701.642826:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775765701.642828:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765701.642833:0:73045:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3079
00000100:00080000:1.0:1775765701.642837:0:73045:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765701.642841:0:73045:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765701.642852:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775765701.642868:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fed80 x1862022114356096/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765701.642882:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765701.642884:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775765701.642886:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:3.0:1775765701.642893:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114355968:192.168.202.110@tcp:250:kworker.0
00000100:00080000:1.0:1775765701.642895:0:73045:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3079
00000100:00080000:1.0:1775765701.642908:0:73045:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765701.642912:0:73045:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765701.642916:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775765701.642927:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff800 x1862022114356224/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765701.642934:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765701.642937:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775765701.642939:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765701.642942:0:73045:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3079
00000100:00080000:1.0:1775765701.642945:0:73045:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765701.642948:0:73045:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765701.642950:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775765701.642961:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ffb80 x1862022114356352/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765701.642969:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765701.642972:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1775765701.642973:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765701.642976:0:73045:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3079
00000100:00080000:1.0:1775765701.642979:0:73045:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765701.642983:0:73045:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000800:00000100:3.0:1775765701.642990:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000008:00100000:1.0:1775765701.642992:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775765701.643002:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdc00 x1862022114356480/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765701.643008:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765701.643011:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775765701.643013:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765701.643017:0:73045:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3079
00000100:00080000:1.0:1775765701.643022:0:73045:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765701.643027:0:73045:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765701.643033:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775765701.643051:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdf80 x1862022114356608/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775765701.643054:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765701/real 1775765701]  req@ffff9f14ab4fca80 x1862022114355968/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765726 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765701.643098:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765701.643102:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775765701.643104:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765701.643109:0:73045:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3079
00000100:00100000:3.0:1775765701.643114:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fca80 x1862022114355968/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765726 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765701.643117:0:73045:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765701.643124:0:73045:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765701.643138:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00080000:3.0:1775765701.643143:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775765701.643152:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765701.643157:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fd880 x1862022114356736/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775765701.643162:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114355968:192.168.202.110@tcp:250:kworker.0
00000100:00080000:1.0:1775765701.643190:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765701.643193:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775765701.643195:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765701.643200:0:73045:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3079
00000100:00080000:1.0:1775765701.643204:0:73045:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:3.0:1775765701.643205:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114356096:192.168.202.110@tcp:38:kworker.0
00000100:00080000:1.0:1775765701.643217:0:73045:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765701.643222:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775765701.643233:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fce00 x1862022114356864/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:3.0:1775765701.643244:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775765701.643263:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765701/real 1775765701]  req@ffff9f14ab4fed80 x1862022114356096/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765726 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775765701.643278:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fed80 x1862022114356096/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765726 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775765701.643286:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775765701.643290:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775765701.643294:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114356096:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775765701.643302:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114356224:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775765701.643324:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775765701.643336:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765701/real 1775765701]  req@ffff9f14ab4ff800 x1862022114356224/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765726 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775765701.643350:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff800 x1862022114356224/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765726 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775765701.643359:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775765701.643363:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775765701.643367:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114356224:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775765701.643374:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ffb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114356352:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775765701.643397:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775765701.643408:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765701/real 1775765701]  req@ffff9f14ab4ffb80 x1862022114356352/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765726 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775765701.643421:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ffb80 x1862022114356352/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765726 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775765701.643431:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775765701.643434:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775765701.643437:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ffb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114356352:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775765701.643444:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114356480:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775765701.643460:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775765701.643470:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765701/real 1775765701]  req@ffff9f14ab4fdc00 x1862022114356480/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765726 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775765701.643483:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdc00 x1862022114356480/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765726 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775765701.643492:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775765701.643495:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775765701.643498:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114356480:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775765701.643504:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114356608:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775765701.643520:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775765701.643529:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765701/real 1775765701]  req@ffff9f14ab4fdf80 x1862022114356608/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765726 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775765701.643541:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdf80 x1862022114356608/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765726 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775765701.643550:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775765701.643585:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775765701.643589:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114356608:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775765701.643596:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114356736:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775765701.643616:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775765701.643626:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765701/real 1775765701]  req@ffff9f14ab4fd880 x1862022114356736/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765726 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775765701.643639:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fd880 x1862022114356736/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765726 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775765701.643649:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775765701.643652:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775765701.643655:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114356736:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775765701.643676:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114356864:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775765701.643694:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775765701.643704:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765701/real 1775765701]  req@ffff9f14ab4fce00 x1862022114356864/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765726 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775765701.643716:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fce00 x1862022114356864/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765726 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775765701.643725:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775765701.643728:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775765701.643732:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114356864:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775765701.898683:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765702.666661:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775765702.922721:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765703.690738:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:2.0:1775765704.714692:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775765704.970747:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765705.738722:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775765705.994739:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765706.762647:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:3.0:1775765706.762904:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114356992:192.168.202.110@tcp:250:kworker.0
00000800:00000100:3.0:1775765706.762961:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775765706.763000:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765706/real 1775765706]  req@ffff9f14a04d2300 x1862022114356992/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765731 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775765706.763032:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d2300 x1862022114356992/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765731 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775765706.763046:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775765706.763056:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775765706.763063:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114356992:192.168.202.110@tcp:250:kworker.0
00000100:00100000:3.0:1775765706.763090:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114357120:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775765706.763118:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775765706.763131:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765706/real 1775765706]  req@ffff9f14a04d1500 x1862022114357120/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765731 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775765706.763169:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1500 x1862022114357120/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765731 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775765706.763185:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775765706.763192:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775765706.763199:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114357120:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775765706.763207:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114357248:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775765706.763235:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775765706.763252:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765706/real 1775765706]  req@ffff9f14a04d0000 x1862022114357248/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765731 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775765706.763283:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0000 x1862022114357248/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765731 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775765706.763305:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775765706.763312:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775765706.763316:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114357248:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775765706.763335:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114357376:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775765706.763367:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775765706.763381:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765706/real 1775765706]  req@ffff9f14a04d3480 x1862022114357376/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765731 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775765706.763403:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3480 x1862022114357376/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765731 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775765706.763416:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775765706.763421:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775765706.763427:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114357376:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775765706.763444:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114357504:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775765706.763461:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775765706.763471:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765706/real 1775765706]  req@ffff9f14a04d1180 x1862022114357504/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765731 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775765706.763497:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1180 x1862022114357504/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765731 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775765706.763502:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775765706.763505:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775765706.763507:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114357504:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775765706.763510:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114357632:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775765706.763520:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775765706.763526:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765706/real 1775765706]  req@ffff9f14a04d2300 x1862022114357632/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765731 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775765706.763533:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d2300 x1862022114357632/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765731 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775765706.763558:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775765706.763560:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775765706.763563:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114357632:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775765706.763567:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114357760:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775765706.763578:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775765706.763609:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765706/real 1775765706]  req@ffff9f14a04d3800 x1862022114357760/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765731 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775765706.763619:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3800 x1862022114357760/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765731 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775765706.763625:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775765706.763627:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775765706.763630:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114357760:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775765706.763641:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114357888:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775765706.763661:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775765706.763689:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765706/real 1775765706]  req@ffff9f14a04d0e00 x1862022114357888/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765731 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775765706.763702:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0e00 x1862022114357888/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765731 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775765706.763710:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775765706.763713:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775765706.763717:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114357888:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775765707.018703:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765707.786697:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775765708.042711:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765708.810716:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775765709.066780:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765709.834715:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775765710.090821:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765710.858656:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:2.0:1775765711.882746:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:3.0:1775765711.882937:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114358016:192.168.202.110@tcp:250:kworker.0
00000800:00000100:3.0:1775765711.883035:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775765711.883105:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765711/real 1775765711]  req@ffff9f14a04d2a00 x1862022114358016/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765736 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775765711.883134:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d2a00 x1862022114358016/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765736 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775765711.883156:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775765711.883176:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775765711.883186:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114358016:192.168.202.110@tcp:250:kworker.0
00000100:00100000:3.0:1775765711.883197:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114358144:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775765711.883224:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775765711.883236:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765711/real 1775765711]  req@ffff9f14a04d0380 x1862022114358144/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765736 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775765711.883247:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0380 x1862022114358144/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765736 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775765711.883254:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775765711.883256:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775765711.883259:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114358144:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775765711.883284:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114358272:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775765711.883301:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775765711.883308:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765711/real 1775765711]  req@ffff9f14a04d1f80 x1862022114358272/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765736 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775765711.883318:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1f80 x1862022114358272/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765736 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775765711.883325:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775765711.883327:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775765711.883330:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114358272:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775765711.883335:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114358400:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775765711.883347:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775765711.883354:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765711/real 1775765711]  req@ffff9f14a04d3b80 x1862022114358400/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765736 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775765711.883363:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3b80 x1862022114358400/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765736 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775765711.883370:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775765711.883372:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775765711.883374:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114358400:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775765711.883378:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114358528:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775765711.883390:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775765711.883397:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765711/real 1775765711]  req@ffff9f14a04d3100 x1862022114358528/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765736 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775765711.883412:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3100 x1862022114358528/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765736 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775765711.883418:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775765711.883420:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775765711.883423:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114358528:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775765711.883426:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114358656:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775765711.883438:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775765711.883445:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765711/real 1775765711]  req@ffff9f14a04d0a80 x1862022114358656/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765736 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775765711.883454:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0a80 x1862022114358656/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765736 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775765711.883461:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775765711.883463:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775765711.883465:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114358656:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775765711.883468:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114358784:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775765711.883481:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775765711.883487:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765711/real 1775765711]  req@ffff9f14a04d0e00 x1862022114358784/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765736 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775765711.883496:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0e00 x1862022114358784/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765736 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775765711.883508:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775765711.883510:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775765711.883513:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114358784:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775765711.883517:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114358912:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775765711.883528:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775765711.883535:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765711/real 1775765711]  req@ffff9f14a04d3800 x1862022114358912/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765736 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775765711.883543:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3800 x1862022114358912/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765736 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775765711.883549:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775765711.883551:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775765711.883584:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114358912:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775765712.138846:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765712.906629:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:2.0:1775765713.930713:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775765714.186795:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765714.954651:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775765715.210817:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765715.978643:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775765716.234747:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765717.002623:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:3.0:1775765717.002852:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114359040:192.168.202.110@tcp:250:kworker.0
00000800:00000100:3.0:1775765717.002943:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775765717.002992:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765717/real 1775765717]  req@ffff9f14a04d2300 x1862022114359040/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765742 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775765717.003025:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d2300 x1862022114359040/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765742 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775765717.003043:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775765717.003051:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775765717.003060:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114359040:192.168.202.110@tcp:250:kworker.0
00000100:00100000:3.0:1775765717.003068:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114359168:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775765717.003106:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775765717.003117:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765717/real 1775765717]  req@ffff9f14a04d1180 x1862022114359168/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765742 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775765717.003127:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1180 x1862022114359168/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765742 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775765717.003134:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775765717.003137:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775765717.003139:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114359168:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775765717.003179:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114359296:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775765717.003193:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775765717.003200:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765717/real 1775765717]  req@ffff9f14a04d3480 x1862022114359296/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765742 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775765717.003209:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3480 x1862022114359296/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765742 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775765717.003215:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775765717.003217:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775765717.003219:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114359296:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775765717.003223:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114359424:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775765717.003235:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775765717.003242:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765717/real 1775765717]  req@ffff9f14a04d0000 x1862022114359424/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765742 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775765717.003251:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0000 x1862022114359424/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765742 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775765717.003257:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775765717.003259:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775765717.003261:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114359424:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775765717.003265:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114359552:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775765717.003275:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775765717.003281:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765717/real 1775765717]  req@ffff9f14a04d1500 x1862022114359552/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765742 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775765717.003288:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1500 x1862022114359552/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765742 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775765717.003293:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775765717.003296:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775765717.003298:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114359552:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775765717.003301:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114359680:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775765717.003312:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775765717.003318:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765717/real 1775765717]  req@ffff9f14a04d3800 x1862022114359680/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765742 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775765717.003325:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3800 x1862022114359680/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765742 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775765717.003332:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775765717.003334:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775765717.003337:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114359680:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775765717.003340:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114359808:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775765717.003367:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775765717.003372:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765717/real 1775765717]  req@ffff9f14a04d0e00 x1862022114359808/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765742 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775765717.003378:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0e00 x1862022114359808/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765742 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775765717.003383:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775765717.003392:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775765717.003394:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114359808:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775765717.003398:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114359936:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775765717.003410:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775765717.003416:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765717/real 1775765717]  req@ffff9f14a04d0a80 x1862022114359936/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765742 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775765717.003423:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0a80 x1862022114359936/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765742 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775765717.003428:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775765717.003430:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775765717.003433:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114359936:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775765717.258763:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765718.026768:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:2.0:1775765719.050741:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775765719.306753:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765720.074732:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:2.0:1775765721.098856:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775765721.354756:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765722.122783:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:3.0:1775765722.122980:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114360064:192.168.202.110@tcp:250:kworker.0
00000800:00000100:3.0:1775765722.123092:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775765722.123164:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765722/real 1775765722]  req@ffff9f14a04d3100 x1862022114360064/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765747 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775765722.123214:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3100 x1862022114360064/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765747 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775765722.123241:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775765722.123252:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775765722.123258:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114360064:192.168.202.110@tcp:250:kworker.0
00000100:00100000:3.0:1775765722.123274:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114360192:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775765722.123313:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775765722.123339:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765722/real 1775765722]  req@ffff9f14a04d3b80 x1862022114360192/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765747 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775765722.123368:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3b80 x1862022114360192/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765747 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775765722.123423:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775765722.123429:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775765722.123435:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114360192:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775765722.123468:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114360320:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775765722.123491:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775765722.123504:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765722/real 1775765722]  req@ffff9f14a04d1f80 x1862022114360320/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765747 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775765722.123518:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1f80 x1862022114360320/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765747 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775765722.123527:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775765722.123554:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775765722.123558:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114360320:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775765722.123565:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114360448:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775765722.123695:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775765722.123710:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765722/real 1775765722]  req@ffff9f14a04d0380 x1862022114360448/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765747 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775765722.123725:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0380 x1862022114360448/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765747 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775765722.123734:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775765722.123738:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775765722.123742:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114360448:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775765722.123749:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114360576:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775765722.123768:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775765722.123778:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765722/real 1775765722]  req@ffff9f14a04d2a00 x1862022114360576/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765747 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775765722.123790:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d2a00 x1862022114360576/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765747 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775765722.123800:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775765722.123803:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775765722.123807:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114360576:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775765722.123812:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114360704:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775765722.123828:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775765722.123837:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765722/real 1775765722]  req@ffff9f14a04d0a80 x1862022114360704/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765747 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775765722.123850:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0a80 x1862022114360704/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765747 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775765722.123859:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775765722.123862:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775765722.123866:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114360704:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775765722.123872:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114360832:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775765722.123888:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775765722.123898:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765722/real 1775765722]  req@ffff9f14a04d0e00 x1862022114360832/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765747 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775765722.123910:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0e00 x1862022114360832/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765747 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775765722.123919:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775765722.123923:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775765722.123930:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114360832:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775765722.123940:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114360960:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775765722.123971:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775765722.123989:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765722/real 1775765722]  req@ffff9f14a04d3800 x1862022114360960/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765747 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775765722.124013:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3800 x1862022114360960/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765747 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775765722.124053:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775765722.124059:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775765722.124065:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114360960:192.168.202.110@tcp:8:kworker.0
00010000:00010000:3.0:1775765722.378862:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765723.146757:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775765723.402800:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765724.170644:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775765724.426784:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765725.194723:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:2.0:1775765726.218693:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775765726.474816:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765727.242671:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:1.0:1775765727.243022:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114361088:192.168.202.110@tcp:250:kworker.0
00000800:00000100:1.0:1775765727.243114:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765727.243169:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765727/real 1775765727]  req@ffff9f14a04d1500 x1862022114361088/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765752 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765727.243199:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1500 x1862022114361088/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765752 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765727.243228:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765727.243238:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765727.243251:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114361088:192.168.202.110@tcp:250:kworker.0
00000100:00100000:1.0:1775765727.243273:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114361216:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765727.243334:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765727.243366:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765727/real 1775765727]  req@ffff9f14a04d0000 x1862022114361216/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765752 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765727.243408:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0000 x1862022114361216/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765752 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765727.243444:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765727.243458:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765727.243468:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114361216:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765727.243519:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114361344:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765727.243650:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765727.243678:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765727/real 1775765727]  req@ffff9f14a04d3480 x1862022114361344/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765752 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765727.243699:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3480 x1862022114361344/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765752 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765727.243710:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765727.243716:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765727.243720:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114361344:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765727.243728:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114361472:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765727.243750:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765727.243761:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765727/real 1775765727]  req@ffff9f14a04d1180 x1862022114361472/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765752 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765727.243774:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1180 x1862022114361472/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765752 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765727.243790:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765727.243793:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765727.243796:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114361472:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775765727.243802:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114361600:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765727.243817:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765727.243825:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765727/real 1775765727]  req@ffff9f14a04d2300 x1862022114361600/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765752 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765727.243837:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d2300 x1862022114361600/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765752 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765727.243845:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765727.243848:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765727.243851:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114361600:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775765727.243856:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114361728:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765727.243871:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765727.243880:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765727/real 1775765727]  req@ffff9f14a04d3800 x1862022114361728/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765752 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765727.243890:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3800 x1862022114361728/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765752 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765727.243898:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765727.243901:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765727.243905:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114361728:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765727.243911:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114361856:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765727.243927:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765727.243935:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765727/real 1775765727]  req@ffff9f14a04d0e00 x1862022114361856/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765752 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765727.243947:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0e00 x1862022114361856/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765752 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765727.243956:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765727.243959:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765727.243962:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114361856:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765727.243983:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114361984:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765727.243999:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765727.244008:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765727/real 1775765727]  req@ffff9f14a04d0a80 x1862022114361984/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765752 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765727.244019:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0a80 x1862022114361984/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765752 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765727.244028:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765727.244031:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765727.244034:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114361984:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775765728.266677:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775765728.522760:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765729.290711:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775765729.546751:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765730.314651:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775765730.570705:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765731.338690:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775765731.594782:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765732.362653:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775765732.362656:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765732.362663:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775765732.362666:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765732.362674:0:8:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3109
00000100:00080000:3.0:1775765732.362683:0:8:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765732.362689:0:8:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775765732.362720:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747480 x1862022114362112/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765732.362742:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765732.362746:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775765732.362748:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765732.362751:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3109
00000100:00080000:3.0:1775765732.362753:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765732.362756:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765732.362764:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775765732.362774:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747800 x1862022114362240/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:1.0:1775765732.362779:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114362112:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775765732.362786:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765732.362789:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775765732.362790:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765732.362793:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3109
00000100:00080000:3.0:1775765732.362795:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765732.362796:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765732.362800:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775765732.362807:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744000 x1862022114362368/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765732.362813:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765732.362815:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775765732.362816:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765732.362819:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3109
00000100:00080000:3.0:1775765732.362822:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765732.362824:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765732.362827:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775765732.362835:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745f80 x1862022114362496/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765732.362841:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765732.362843:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775765732.362844:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765732.362857:0:8:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3109
00000100:00080000:3.0:1775765732.362859:0:8:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765732.362861:0:8:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765732.362864:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775765732.362870:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744700 x1862022114362624/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765732.362877:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765732.362879:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775765732.362881:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765732.362884:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3109
00000100:00080000:3.0:1775765732.362887:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:1.0:1775765732.362891:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114362240:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775765732.362892:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765732.362898:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775765732.362909:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747100 x1862022114362752/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765732.362927:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765732.362930:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775765732.362932:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:1.0:1775765732.362936:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114362368:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775765732.362936:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3109
00000100:00080000:3.0:1775765732.362944:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765732.362947:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765732.362950:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775765732.362958:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745180 x1862022114362880/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765732.362974:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765732.362976:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775765732.362977:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:1.0:1775765732.362980:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114362496:192.168.202.110@tcp:8:kworker.0
00000100:00080000:3.0:1775765732.362981:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3109
00000100:00080000:3.0:1775765732.362987:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765732.362991:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765732.362994:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775765732.363004:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745c00 x1862022114363008/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:1.0:1775765732.363021:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114362624:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775765732.363051:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114362752:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765732.363071:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114362880:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765732.363090:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114363008:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775765733.386689:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775765733.642886:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765734.410673:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:2.0:1775765735.434859:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:1.0:1775765735.435305:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765732/real 1775765735]  req@ffff9f1485747100 x1862022114362752/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765757 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765735.435346:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747100 x1862022114362752/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765757 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765735.435364:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765735.435372:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765735.435380:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114362752:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765735.435391:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765732/real 1775765735]  req@ffff9f1485745180 x1862022114362880/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765757 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765735.435416:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745180 x1862022114362880/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765757 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765735.435429:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765735.435435:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765735.435441:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114362880:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765735.435452:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765732/real 1775765735]  req@ffff9f1485745c00 x1862022114363008/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765757 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765735.435472:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745c00 x1862022114363008/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765757 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765735.435484:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765735.435512:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765735.435518:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114363008:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775765735.435531:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765732/real 1775765735]  req@ffff9f1485747800 x1862022114362240/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765757 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765735.435550:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747800 x1862022114362240/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765757 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765735.435611:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765735.435618:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765735.435625:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114362240:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765735.435637:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765732/real 1775765735]  req@ffff9f1485744000 x1862022114362368/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765757 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765735.435658:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744000 x1862022114362368/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765757 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765735.435671:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765735.435678:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765735.435687:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114362368:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765735.435700:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765732/real 1775765735]  req@ffff9f1485745f80 x1862022114362496/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765757 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765735.435727:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745f80 x1862022114362496/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765757 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765735.435742:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765735.435750:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765735.435758:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114362496:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775765735.435769:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765732/real 1775765735]  req@ffff9f1485744700 x1862022114362624/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765757 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765735.435789:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744700 x1862022114362624/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765757 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765735.435801:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765735.435807:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765735.435811:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114362624:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775765735.435821:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765732/real 1775765735]  req@ffff9f1485747480 x1862022114362112/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765757 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765735.435840:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747480 x1862022114362112/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765757 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765735.435851:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765735.435856:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765735.435862:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114362112:192.168.202.110@tcp:250:kworker.0
00010000:00010000:2.0:1775765735.690729:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765736.458664:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775765736.714673:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1775765737.482716:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765737.482727:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00100000:2.0:1775765737.482727:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:1.0:1775765737.482732:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765737.482744:0:73045:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3114
00000100:00080000:1.0:1775765737.482757:0:73045:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765737.482765:0:73045:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775765737.482826:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fce00 x1862022114363136/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765737.482868:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765737.482872:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775765737.482874:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765737.482879:0:73045:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3114
00000100:00080000:1.0:1775765737.482884:0:73045:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765737.482888:0:73045:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765737.482900:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775765737.482916:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fd880 x1862022114363264/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765737.482948:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765737.482953:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775765737.482955:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765737.482961:0:73045:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3114
00000100:00080000:1.0:1775765737.482965:0:73045:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765737.482972:0:73045:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775765737.482975:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114363136:192.168.202.110@tcp:250:kworker.0
00000008:00100000:1.0:1775765737.482980:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775765737.482996:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdf80 x1862022114363392/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765737.483009:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765737.483013:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775765737.483017:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765737.483025:0:73045:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3114
00000100:00080000:1.0:1775765737.483030:0:73045:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765737.483037:0:73045:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765737.483044:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775765737.483057:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdc00 x1862022114363520/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765737.483068:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765737.483071:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1775765737.483074:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765737.483081:0:73045:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3114
00000800:00000100:3.0:1775765737.483082:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:1.0:1775765737.483087:0:73045:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765737.483093:0:73045:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765737.483100:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775765737.483115:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ffb80 x1862022114363648/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765737.483127:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765737.483132:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775765737.483135:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:3.0:1775765737.483141:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765737/real 1775765737]  req@ffff9f14ab4fce00 x1862022114363136/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765762 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765737.483145:0:73045:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3114
00000100:00080000:1.0:1775765737.483154:0:73045:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765737.483164:0:73045:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765737.483171:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775765737.483177:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fce00 x1862022114363136/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765762 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765737.483191:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff800 x1862022114363776/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765737.483214:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775765737.483226:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:1.0:1775765737.483228:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765737.483236:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00100000:3.0:1775765737.483236:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114363136:192.168.202.110@tcp:250:kworker.0
00000100:00080000:1.0:1775765737.483242:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:3.0:1775765737.483253:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114363264:192.168.202.110@tcp:38:kworker.0
00000100:00080000:1.0:1775765737.483265:0:73045:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3114
00000100:00080000:1.0:1775765737.483271:0:73045:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765737.483278:0:73045:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765737.483284:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000800:00000100:3.0:1775765737.483291:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765737.483303:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fed80 x1862022114363904/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775765737.483313:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765737/real 1775765737]  req@ffff9f14ab4fd880 x1862022114363264/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765762 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765737.483319:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765737.483329:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775765737.483333:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765737.483341:0:73045:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3114
00000100:00080000:1.0:1775765737.483346:0:73045:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:3.0:1775765737.483347:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fd880 x1862022114363264/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765762 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765737.483356:0:73045:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765737.483369:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00080000:3.0:1775765737.483371:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775765737.483380:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765737.483387:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fca80 x1862022114364032/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775765737.483388:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114363264:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775765737.483442:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114363392:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775765737.483490:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775765737.483505:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765737/real 1775765737]  req@ffff9f14ab4fdf80 x1862022114363392/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765762 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775765737.483521:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdf80 x1862022114363392/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765762 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775765737.483531:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775765737.483535:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775765737.483539:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114363392:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775765737.483547:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114363520:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775765737.483600:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775765737.483987:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765737/real 1775765737]  req@ffff9f14ab4fdc00 x1862022114363520/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765762 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775765737.484036:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdc00 x1862022114363520/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765762 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775765737.484050:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775765737.484055:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775765737.484060:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114363520:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775765737.484087:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ffb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114363648:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775765737.484152:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775765737.484177:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765737/real 1775765737]  req@ffff9f14ab4ffb80 x1862022114363648/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765762 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775765737.484191:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ffb80 x1862022114363648/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765762 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775765737.484197:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775765737.484210:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775765737.484213:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ffb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114363648:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775765737.484221:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114363776:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775765737.484246:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775765737.484258:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765737/real 1775765737]  req@ffff9f14ab4ff800 x1862022114363776/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765762 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775765737.484282:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff800 x1862022114363776/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765762 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775765737.484290:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775765737.484293:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775765737.484296:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114363776:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775765737.484302:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114363904:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775765737.484323:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775765737.484332:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765737/real 1775765737]  req@ffff9f14ab4fed80 x1862022114363904/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765762 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775765737.484340:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fed80 x1862022114363904/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765762 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775765737.484345:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775765737.484347:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775765737.484349:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114363904:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775765737.484353:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114364032:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775765737.484366:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775765737.484372:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765737/real 1775765737]  req@ffff9f14ab4fca80 x1862022114364032/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765762 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775765737.484379:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fca80 x1862022114364032/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765762 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775765737.484389:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775765737.484391:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775765737.484393:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114364032:192.168.202.110@tcp:8:kworker.0
00010000:00010000:3.0:1775765737.738692:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765738.506829:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775765738.762796:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765739.530653:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:2.0:1775765740.554669:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775765740.810834:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765741.578629:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:1.0:1775765742.602632:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765742.602639:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00100000:2.0:1775765742.602639:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:1.0:1775765742.602643:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765742.602651:0:73045:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3120
00000100:00080000:1.0:1775765742.602660:0:73045:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765742.602666:0:73045:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775765742.602701:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fe300 x1862022114364160/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765742.602724:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765742.602726:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775765742.602727:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765742.602730:0:73045:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3120
00000100:00080000:1.0:1775765742.602732:0:73045:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765742.602735:0:73045:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765742.602743:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775765742.602752:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff100 x1862022114364288/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765742.602759:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765742.602762:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775765742.602763:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765742.602766:0:73045:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3120
00000100:00080000:1.0:1775765742.602769:0:73045:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00100000:3.0:1775765742.602773:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114364160:192.168.202.110@tcp:250:kworker.0
00000100:00080000:1.0:1775765742.602775:0:73045:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765742.602782:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775765742.602789:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fea00 x1862022114364416/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765742.602795:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765742.602797:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775765742.602798:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765742.602800:0:73045:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3120
00000100:00080000:1.0:1775765742.602802:0:73045:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765742.602804:0:73045:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765742.602808:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775765742.602815:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fc380 x1862022114364544/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765742.602830:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765742.602833:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1775765742.602835:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765742.602839:0:73045:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3120
00000100:00080000:1.0:1775765742.602843:0:73045:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765742.602846:0:73045:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765742.602852:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:3.0:1775765742.602853:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765742.602867:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fca80 x1862022114364672/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765742.602884:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765742.602887:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775765742.602888:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765742.602903:0:73045:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3120
00000100:00080000:1.0:1775765742.602907:0:73045:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765742.602912:0:73045:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775765742.602912:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765742/real 1775765742]  req@ffff9f14ab4fe300 x1862022114364160/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765767 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:1.0:1775765742.602919:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775765742.602932:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fed80 x1862022114364800/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765742.602938:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765742.602942:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00100000:3.0:1775765742.602943:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fe300 x1862022114364160/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765767 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765742.602944:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765742.602950:0:73045:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3120
00000100:00080000:1.0:1775765742.602955:0:73045:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765742.602960:0:73045:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765742.602966:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00080000:3.0:1775765742.602968:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00100000:1.0:1775765742.602974:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff800 x1862022114364928/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765742.602974:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775765742.602982:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114364160:192.168.202.110@tcp:250:kworker.0
00000100:00080000:1.0:1775765742.602991:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:3.0:1775765742.602994:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114364288:192.168.202.110@tcp:38:kworker.0
00000100:00080000:1.0:1775765742.602995:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775765742.602998:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765742.603004:0:73045:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3120
00000100:00080000:1.0:1775765742.603008:0:73045:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765742.603010:0:73045:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765742.603014:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775765742.603023:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ffb80 x1862022114365056/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:3.0:1775765742.603026:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775765742.603041:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765742/real 1775765742]  req@ffff9f14ab4ff100 x1862022114364288/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765767 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775765742.603051:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff100 x1862022114364288/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765767 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775765742.603057:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775765742.603059:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775765742.603062:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114364288:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775765742.603078:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114364416:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775765742.603090:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775765742.603095:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765742/real 1775765742]  req@ffff9f14ab4fea00 x1862022114364416/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765767 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775765742.603103:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fea00 x1862022114364416/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765767 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775765742.603108:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775765742.603110:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775765742.603112:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114364416:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775765742.603115:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114364544:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775765742.603125:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775765742.603130:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765742/real 1775765742]  req@ffff9f14ab4fc380 x1862022114364544/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765767 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775765742.603137:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fc380 x1862022114364544/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765767 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775765742.603142:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775765742.603143:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775765742.603145:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114364544:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775765742.603148:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114364672:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775765742.603157:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775765742.603162:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765742/real 1775765742]  req@ffff9f14ab4fca80 x1862022114364672/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765767 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775765742.603169:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fca80 x1862022114364672/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765767 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775765742.603174:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775765742.603175:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775765742.603177:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114364672:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775765742.603180:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114364800:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775765742.603189:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775765742.603194:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765742/real 1775765742]  req@ffff9f14ab4fed80 x1862022114364800/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765767 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775765742.603201:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fed80 x1862022114364800/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765767 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775765742.603206:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775765742.603208:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775765742.603209:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114364800:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775765742.603212:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114364928:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775765742.603221:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775765742.603226:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765742/real 1775765742]  req@ffff9f14ab4ff800 x1862022114364928/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765767 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775765742.603233:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff800 x1862022114364928/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765767 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775765742.603250:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775765742.603252:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775765742.603254:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114364928:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775765742.603257:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ffb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114365056:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775765742.603265:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775765742.603270:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765742/real 1775765742]  req@ffff9f14ab4ffb80 x1862022114365056/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765767 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775765742.603276:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ffb80 x1862022114365056/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765767 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775765742.603281:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775765742.603283:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775765742.603285:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ffb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114365056:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775765742.858740:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765743.626650:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775765743.882704:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765744.650645:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775765744.906792:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765745.674633:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775765745.930756:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765746.698714:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:2.0:1775765747.722659:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:3.0:1775765747.722889:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114365184:192.168.202.110@tcp:250:kworker.0
00000800:00000100:3.0:1775765747.722969:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775765747.723021:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765747/real 1775765747]  req@ffff9f14a04d2a00 x1862022114365184/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765772 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775765747.723050:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d2a00 x1862022114365184/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765772 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775765747.723076:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775765747.723090:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775765747.723097:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114365184:192.168.202.110@tcp:250:kworker.0
00000100:00100000:3.0:1775765747.723112:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114365312:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775765747.723149:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775765747.723165:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765747/real 1775765747]  req@ffff9f14a04d0380 x1862022114365312/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765772 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775765747.723190:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0380 x1862022114365312/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765772 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775765747.723199:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775765747.723203:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775765747.723208:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114365312:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775765747.723234:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114365440:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775765747.723250:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775765747.723258:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765747/real 1775765747]  req@ffff9f14a04d1f80 x1862022114365440/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765772 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775765747.723269:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1f80 x1862022114365440/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765772 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775765747.723276:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775765747.723279:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775765747.723281:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114365440:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775765747.723286:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114365568:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775765747.723301:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775765747.723308:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765747/real 1775765747]  req@ffff9f14a04d3b80 x1862022114365568/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765772 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775765747.723318:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3b80 x1862022114365568/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765772 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775765747.723325:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775765747.723328:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775765747.723330:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114365568:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775765747.723335:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114365696:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775765747.723348:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775765747.723355:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765747/real 1775765747]  req@ffff9f14a04d3100 x1862022114365696/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765772 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775765747.723365:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3100 x1862022114365696/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765772 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775765747.723402:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775765747.723406:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775765747.723410:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114365696:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775765747.723416:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114365824:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775765747.723436:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775765747.723455:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765747/real 1775765747]  req@ffff9f14a04d0a80 x1862022114365824/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765772 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775765747.723473:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0a80 x1862022114365824/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765772 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775765747.723481:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775765747.723483:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775765747.723486:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114365824:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775765747.723490:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114365952:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775765747.723503:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775765747.723510:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765747/real 1775765747]  req@ffff9f14a04d0e00 x1862022114365952/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765772 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775765747.723520:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0e00 x1862022114365952/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765772 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775765747.723527:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775765747.723529:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775765747.723532:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114365952:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775765747.723537:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114366080:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775765747.723549:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775765747.723587:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765747/real 1775765747]  req@ffff9f14a04d3800 x1862022114366080/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765772 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775765747.723598:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3800 x1862022114366080/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765772 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775765747.723611:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775765747.723614:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775765747.723617:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114366080:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775765747.978700:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765748.746668:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:2.0:1775765749.770695:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775765750.026886:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765750.794705:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775765751.050806:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765751.818616:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775765752.074721:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765752.842665:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775765752.842697:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765752.842705:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775765752.842710:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765752.842719:0:8:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3130
00000100:00080000:3.0:1775765752.842730:0:8:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765752.842737:0:8:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775765752.842776:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745500 x1862022114366208/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765752.842807:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765752.842811:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775765752.842813:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765752.842817:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3130
00000100:00080000:3.0:1775765752.842820:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765752.842823:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765752.842831:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775765752.842843:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744380 x1862022114366336/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765752.842871:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765752.842876:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775765752.842878:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765752.842883:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3130
00000100:00100000:1.0:1775765752.842892:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114366208:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775765752.842902:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765752.842909:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765752.842916:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775765752.842945:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746a00 x1862022114366464/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765752.842956:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765752.842959:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775765752.842961:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765752.842964:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3130
00000100:00080000:3.0:1775765752.842970:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765752.842973:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765752.842978:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:1.0:1775765752.842984:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775765752.842992:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746300 x1862022114366592/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765752.843002:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765752.843014:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775765752.843016:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765752.843019:0:8:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3130
00000100:00080000:3.0:1775765752.843023:0:8:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765752.843026:0:8:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765752.843032:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775765752.843041:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765752/real 1775765752]  req@ffff9f1485745500 x1862022114366208/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765777 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775765752.843044:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747b80 x1862022114366720/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765752.843070:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765752.843079:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775765752.843082:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765752.843085:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3130
00000100:00080000:3.0:1775765752.843088:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765752.843092:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765752.843097:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775765752.843102:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745500 x1862022114366208/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765777 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775765752.843109:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744e00 x1862022114366848/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765752.843143:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765752.843146:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775765752.843147:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775765752.843151:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765752.843161:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3130
00000100:00080000:1.0:1775765752.843162:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:3.0:1775765752.843165:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765752.843171:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775765752.843176:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114366208:192.168.202.110@tcp:250:kworker.0
00000008:00100000:3.0:1775765752.843178:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775765752.843193:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746680 x1862022114366976/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:1.0:1775765752.843198:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114366336:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775765752.843210:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765752.843213:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775765752.843215:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765752.843221:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3130
00000100:00080000:3.0:1775765752.843227:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765752.843233:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765752.843241:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:1.0:1775765752.843265:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775765752.843278:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745880 x1862022114367104/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:1.0:1775765752.843295:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765752/real 1775765752]  req@ffff9f1485744380 x1862022114366336/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765777 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765752.843364:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744380 x1862022114366336/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765777 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765752.843379:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765752.843385:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765752.843392:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114366336:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765752.843423:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114366464:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765752.843467:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765752.843483:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765752/real 1775765752]  req@ffff9f1485746a00 x1862022114366464/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765777 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765752.843501:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746a00 x1862022114366464/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765777 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765752.843513:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765752.843517:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765752.843522:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114366464:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765752.843531:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114366592:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765752.843598:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765752.843614:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765752/real 1775765752]  req@ffff9f1485746300 x1862022114366592/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765777 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765752.843632:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746300 x1862022114366592/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765777 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765752.843646:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765752.843651:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765752.843655:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114366592:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775765752.843664:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114366720:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765752.843691:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765752.843706:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765752/real 1775765752]  req@ffff9f1485747b80 x1862022114366720/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765777 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765752.843723:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747b80 x1862022114366720/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765777 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765752.843735:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765752.843740:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765752.843745:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114366720:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775765752.843753:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114366848:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765752.843779:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765752.843795:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765752/real 1775765752]  req@ffff9f1485744e00 x1862022114366848/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765777 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765752.843813:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744e00 x1862022114366848/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765777 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765752.843825:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765752.843830:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765752.843835:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114366848:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765752.843843:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114366976:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765752.843871:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765752.843885:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765752/real 1775765752]  req@ffff9f1485746680 x1862022114366976/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765777 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765752.843902:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746680 x1862022114366976/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765777 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765752.843916:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765752.843920:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765752.843925:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114366976:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765752.843934:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114367104:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765752.843960:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765752.843975:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765752/real 1775765752]  req@ffff9f1485745880 x1862022114367104/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765777 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765752.843992:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745880 x1862022114367104/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765777 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765752.844005:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765752.844010:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765752.844014:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114367104:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775765753.098838:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765753.866636:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:2.0:1775765754.890621:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775765755.146841:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765755.914659:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:2.0:1775765756.938624:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775765757.194690:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765757.962624:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:1.0:1775765757.962778:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114367232:192.168.202.110@tcp:250:kworker.0
00000800:00000100:1.0:1775765757.962846:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765757.962892:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765757/real 1775765757]  req@ffff9f14a04d2300 x1862022114367232/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765782 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765757.962928:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d2300 x1862022114367232/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765782 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765757.962959:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765757.962967:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765757.962978:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114367232:192.168.202.110@tcp:250:kworker.0
00000100:00100000:1.0:1775765757.963010:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114367360:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765757.963044:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765757.963064:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765757/real 1775765757]  req@ffff9f14a04d1180 x1862022114367360/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765782 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765757.963077:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1180 x1862022114367360/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765782 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765757.963085:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765757.963088:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765757.963092:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114367360:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765757.963098:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114367488:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765757.963118:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765757.963130:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765757/real 1775765757]  req@ffff9f14a04d3480 x1862022114367488/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765782 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765757.963153:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3480 x1862022114367488/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765782 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765757.963162:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765757.963165:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765757.963168:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114367488:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765757.963174:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114367616:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765757.963192:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765757.963202:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765757/real 1775765757]  req@ffff9f14a04d0000 x1862022114367616/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765782 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765757.963213:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0000 x1862022114367616/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765782 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765757.963220:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765757.963223:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765757.963227:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114367616:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775765757.963232:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114367744:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765757.963247:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765757.963256:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765757/real 1775765757]  req@ffff9f14a04d1500 x1862022114367744/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765782 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765757.963267:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1500 x1862022114367744/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765782 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765757.963293:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765757.963296:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765757.963299:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114367744:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775765757.963305:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114367872:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765757.963325:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765757.963334:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765757/real 1775765757]  req@ffff9f14a04d3800 x1862022114367872/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765782 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765757.963346:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3800 x1862022114367872/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765782 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765757.963359:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765757.963362:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765757.963365:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114367872:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765757.963383:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114368000:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765757.963402:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765757.963411:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765757/real 1775765757]  req@ffff9f14a04d0e00 x1862022114368000/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765782 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765757.963421:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0e00 x1862022114368000/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765782 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765757.963428:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765757.963431:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765757.963434:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114368000:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765757.963439:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114368128:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765757.963452:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765757.963460:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765757/real 1775765757]  req@ffff9f14a04d0a80 x1862022114368128/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765782 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765757.963470:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0a80 x1862022114368128/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765782 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765757.963477:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765757.963480:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765757.963483:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114368128:192.168.202.110@tcp:8:kworker.0
00010000:00010000:3.0:1775765758.218682:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765758.986637:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775765759.242697:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765760.010631:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775765760.266712:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765761.034653:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:2.0:1775765762.058680:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775765762.314705:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765763.082634:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:1.0:1775765763.082777:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114368256:192.168.202.110@tcp:250:kworker.0
00000800:00000100:1.0:1775765763.082833:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765763.082886:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765763/real 1775765763]  req@ffff9f14a04d3100 x1862022114368256/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765788 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765763.082920:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3100 x1862022114368256/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765788 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765763.082932:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765763.082939:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765763.082943:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114368256:192.168.202.110@tcp:250:kworker.0
00000100:00100000:1.0:1775765763.082969:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114368384:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765763.083000:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765763.083015:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765763/real 1775765763]  req@ffff9f14a04d3b80 x1862022114368384/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765788 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765763.083038:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3b80 x1862022114368384/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765788 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765763.083061:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765763.083070:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765763.083080:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114368384:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765763.083096:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114368512:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765763.083132:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765763.083145:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765763/real 1775765763]  req@ffff9f14a04d1f80 x1862022114368512/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765788 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765763.083153:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1f80 x1862022114368512/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765788 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765763.083159:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765763.083162:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765763.083165:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114368512:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765763.083169:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114368640:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765763.083193:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765763.083202:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765763/real 1775765763]  req@ffff9f14a04d0380 x1862022114368640/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765788 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765763.083212:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0380 x1862022114368640/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765788 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765763.083221:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765763.083224:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765763.083241:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114368640:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775765763.083257:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114368768:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765763.083272:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765763.083281:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765763/real 1775765763]  req@ffff9f14a04d2a00 x1862022114368768/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765788 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765763.083289:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d2a00 x1862022114368768/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765788 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765763.083295:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765763.083298:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765763.083302:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114368768:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775765763.083306:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114368896:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765763.083319:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765763.083332:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765763/real 1775765763]  req@ffff9f14a04d3100 x1862022114368896/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765788 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765763.083342:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3100 x1862022114368896/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765788 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765763.083352:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765763.083355:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765763.083356:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114368896:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765763.083360:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114369024:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765763.083376:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765763.083396:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765763/real 1775765763]  req@ffff9f14a04d0a80 x1862022114369024/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765788 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765763.083416:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0a80 x1862022114369024/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765788 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765763.083423:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765763.083426:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765763.083430:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114369024:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765763.083436:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114369152:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765763.083448:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765763.083453:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765763/real 1775765763]  req@ffff9f14a04d0e00 x1862022114369152/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765788 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765763.083460:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0e00 x1862022114369152/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765788 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765763.083465:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765763.083466:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765763.083468:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114369152:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775765764.106666:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775765764.362713:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765765.130614:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775765765.386756:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765766.154685:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775765766.410671:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765767.178660:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775765767.434704:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765768.202732:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:2.0:1775765768.202782:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765768.202788:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1775765768.202793:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765768.202804:0:73045:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3145
00000100:00080000:2.0:1775765768.202815:0:73045:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765768.202823:0:73045:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775765768.202883:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1180 x1862022114369280/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775765768.202913:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765768.202916:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775765768.202918:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765768.202923:0:73045:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3145
00000100:00080000:2.0:1775765768.202926:0:73045:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765768.202930:0:73045:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775765768.202942:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775765768.202954:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1f80 x1862022114369408/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775765768.202967:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765768.202971:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00100000:1.0:1775765768.202972:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114369280:192.168.202.110@tcp:250:kworker.0
00000100:00080000:2.0:1775765768.202973:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765768.202986:0:73045:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3145
00000100:00080000:2.0:1775765768.202990:0:73045:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765768.202993:0:73045:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775765768.202999:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775765768.203013:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2a00 x1862022114369536/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775765768.203023:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765768.203026:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775765768.203028:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765768.203032:0:73045:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3145
00000100:00080000:2.0:1775765768.203036:0:73045:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765768.203039:0:73045:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775765768.203052:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:1.0:1775765768.203062:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775765768.203068:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2680 x1862022114369664/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775765768.203079:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765768.203083:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1775765768.203085:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765768.203089:0:73045:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3145
00000100:00080000:2.0:1775765768.203092:0:73045:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765768.203095:0:73045:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775765768.203100:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775765768.203113:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1500 x1862022114369792/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:1.0:1775765768.203124:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765768/real 1775765768]  req@ffff9f14852d1180 x1862022114369280/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765793 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765768.203126:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765768.203137:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775765768.203140:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765768.203153:0:73045:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3145
00000100:00080000:2.0:1775765768.203161:0:73045:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765768.203166:0:73045:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775765768.203172:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775765768.203199:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1180 x1862022114369280/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765793 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775765768.203205:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0a80 x1862022114369920/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765768.203235:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775765768.203237:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765768.203245:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775765768.203249:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:2.0:1775765768.203249:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765768.203261:0:73045:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3145
00000100:00100000:1.0:1775765768.203263:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114369280:192.168.202.110@tcp:250:kworker.0
00000100:00080000:2.0:1775765768.203267:0:73045:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765768.203274:0:73045:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775765768.203279:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775765768.203290:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0700 x1862022114370048/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:1.0:1775765768.203295:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114369408:192.168.202.110@tcp:38:kworker.0
00000100:00080000:2.0:1775765768.203315:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765768.203319:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775765768.203321:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765768.203327:0:73045:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3145
00000100:00080000:2.0:1775765768.203332:0:73045:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765768.203339:0:73045:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000800:00000100:1.0:1775765768.203340:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000008:00100000:2.0:1775765768.203348:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775765768.203363:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765768/real 1775765768]  req@ffff9f14852d1f80 x1862022114369408/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765793 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775765768.203366:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2d80 x1862022114370176/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:1.0:1775765768.203394:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1f80 x1862022114369408/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765793 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765768.203411:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765768.203415:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765768.203419:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114369408:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765768.203427:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114369536:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765768.203446:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765768.203455:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765768/real 1775765768]  req@ffff9f14852d2a00 x1862022114369536/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765793 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765768.203467:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2a00 x1862022114369536/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765793 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765768.203475:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765768.203477:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765768.203480:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114369536:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765768.203486:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114369664:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765768.203501:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765768.203510:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765768/real 1775765768]  req@ffff9f14852d2680 x1862022114369664/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765793 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765768.203521:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2680 x1862022114369664/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765793 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765768.203601:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765768.203607:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765768.203611:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114369664:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775765768.203619:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114369792:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765768.203665:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765768.203677:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765768/real 1775765768]  req@ffff9f14852d1500 x1862022114369792/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765793 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765768.203689:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1500 x1862022114369792/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765793 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765768.203697:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765768.203700:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765768.203703:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114369792:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775765768.203709:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114369920:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765768.203724:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765768.203736:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765768/real 1775765768]  req@ffff9f14852d0a80 x1862022114369920/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765793 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765768.203755:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0a80 x1862022114369920/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765793 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765768.203767:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765768.203772:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765768.203777:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114369920:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765768.203809:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114370048:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765768.203845:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765768.203864:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765768/real 1775765768]  req@ffff9f14852d0700 x1862022114370048/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765793 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765768.203884:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0700 x1862022114370048/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765793 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765768.203898:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765768.203903:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765768.203908:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114370048:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765768.203918:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114370176:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765768.203951:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765768.203969:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765768/real 1775765768]  req@ffff9f14852d2d80 x1862022114370176/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765793 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765768.203988:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2d80 x1862022114370176/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765793 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765768.204002:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765768.204007:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765768.204012:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114370176:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775765769.226622:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775765769.482704:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765770.250667:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:2.0:1775765771.274646:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775765771.530714:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765772.298695:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775765772.554721:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765773.322642:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775765773.322658:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765773.322663:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775765773.322667:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765773.322675:0:73045:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3150
00000100:00080000:3.0:1775765773.322685:0:73045:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765773.322690:0:73045:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775765773.322729:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744a80 x1862022114370304/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765773.322761:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765773.322764:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775765773.322765:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765773.322768:0:73045:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3150
00000100:00080000:3.0:1775765773.322771:0:73045:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765773.322775:0:73045:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765773.322783:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775765773.322796:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745880 x1862022114370432/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765773.322803:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765773.322806:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775765773.322808:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765773.322812:0:73045:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3150
00000100:00100000:1.0:1775765773.322816:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114370304:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775765773.322816:0:73045:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765773.322821:0:73045:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765773.322827:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775765773.322834:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746680 x1862022114370560/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765773.322841:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765773.322843:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775765773.322845:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765773.322847:0:73045:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3150
00000100:00080000:3.0:1775765773.322850:0:73045:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765773.322852:0:73045:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765773.322855:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775765773.322862:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744e00 x1862022114370688/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765773.322868:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765773.322870:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775765773.322871:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765773.322873:0:73045:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3150
00000100:00080000:3.0:1775765773.322876:0:73045:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000800:00000100:1.0:1775765773.322879:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775765773.322883:0:73045:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765773.322887:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775765773.322894:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747b80 x1862022114370816/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765773.322901:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765773.322903:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775765773.322904:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765773.322906:0:73045:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3150
00000100:00080000:3.0:1775765773.322908:0:73045:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765773.322911:0:73045:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765773.322915:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775765773.322920:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765773/real 1775765773]  req@ffff9f1485744a80 x1862022114370304/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765798 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775765773.322924:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746300 x1862022114370944/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765773.322946:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765773.322948:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775765773.322949:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:1.0:1775765773.322954:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744a80 x1862022114370304/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765798 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775765773.322954:0:73045:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3150
00000100:00080000:1.0:1775765773.322965:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775765773.322965:0:73045:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765773.322969:0:73045:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00080000:1.0:1775765773.322974:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000008:00100000:3.0:1775765773.322976:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775765773.322979:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114370304:192.168.202.110@tcp:250:kworker.0
00000100:00100000:3.0:1775765773.322986:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746a00 x1862022114371072/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:1.0:1775765773.322989:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114370432:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775765773.323002:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765773.323004:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775765773.323005:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765773.323009:0:73045:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3150
00000100:00080000:3.0:1775765773.323016:0:73045:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765773.323018:0:73045:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000800:00000100:1.0:1775765773.323023:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000008:00100000:3.0:1775765773.323024:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775765773.323036:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744380 x1862022114371200/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:1.0:1775765773.323045:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765773/real 1775765773]  req@ffff9f1485745880 x1862022114370432/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765798 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765773.323061:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745880 x1862022114370432/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765798 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765773.323069:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765773.323071:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765773.323074:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114370432:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765773.323092:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114370560:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765773.323107:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765773.323115:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765773/real 1775765773]  req@ffff9f1485746680 x1862022114370560/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765798 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765773.323123:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746680 x1862022114370560/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765798 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765773.323156:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765773.323158:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765773.323161:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114370560:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765773.323165:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114370688:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765773.323180:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765773.323187:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765773/real 1775765773]  req@ffff9f1485744e00 x1862022114370688/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765798 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765773.323195:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744e00 x1862022114370688/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765798 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765773.323200:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765773.323202:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765773.323204:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114370688:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775765773.323207:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114370816:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765773.323216:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765773.323221:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765773/real 1775765773]  req@ffff9f1485747b80 x1862022114370816/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765798 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765773.323228:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747b80 x1862022114370816/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765798 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765773.323233:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765773.323234:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765773.323237:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114370816:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775765773.323241:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114370944:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765773.323250:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765773.323255:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765773/real 1775765773]  req@ffff9f1485746300 x1862022114370944/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765798 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765773.323262:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746300 x1862022114370944/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765798 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765773.323267:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765773.323268:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765773.323270:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114370944:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765773.323273:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114371072:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765773.323282:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765773.323287:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765773/real 1775765773]  req@ffff9f1485746a00 x1862022114371072/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765798 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765773.323293:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746a00 x1862022114371072/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765798 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765773.323301:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765773.323304:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765773.323307:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114371072:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765773.323313:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114371200:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765773.323332:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765773.323343:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765773/real 1775765773]  req@ffff9f1485744380 x1862022114371200/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765798 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765773.323355:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744380 x1862022114371200/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765798 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765773.323363:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765773.323365:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765773.323367:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114371200:192.168.202.110@tcp:8:kworker.0
00010000:00010000:3.0:1775765773.578755:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765774.346628:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775765774.602693:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765775.370635:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:2.0:1775765776.394639:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775765776.650705:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765777.418681:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:2.0:1775765778.442651:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:1.0:1775765778.442810:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114371328:192.168.202.110@tcp:250:kworker.0
00000800:00000100:1.0:1775765778.442879:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765778.442926:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765778/real 1775765778]  req@ffff9f14a04d3800 x1862022114371328/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765803 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765778.442946:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3800 x1862022114371328/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765803 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765778.442964:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765778.442971:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765778.442976:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114371328:192.168.202.110@tcp:250:kworker.0
00000100:00100000:1.0:1775765778.442999:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114371456:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765778.443025:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765778.443038:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765778/real 1775765778]  req@ffff9f14a04d1500 x1862022114371456/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765803 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765778.443053:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1500 x1862022114371456/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765803 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765778.443069:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765778.443074:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765778.443123:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114371456:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765778.443137:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114371584:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765778.443198:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765778.443219:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765778/real 1775765778]  req@ffff9f14a04d0000 x1862022114371584/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765803 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765778.443253:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0000 x1862022114371584/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765803 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765778.443266:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765778.443271:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765778.443276:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114371584:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765778.443283:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114371712:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765778.443303:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765778.443315:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765778/real 1775765778]  req@ffff9f14a04d3480 x1862022114371712/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765803 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765778.443326:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3480 x1862022114371712/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765803 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765778.443331:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765778.443333:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765778.443336:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114371712:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775765778.443348:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114371840:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765778.443358:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765778.443364:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765778/real 1775765778]  req@ffff9f14a04d1180 x1862022114371840/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765803 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765778.443371:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1180 x1862022114371840/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765803 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765778.443376:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765778.443378:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765778.443380:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114371840:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775765778.443383:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114371968:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765778.443392:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765778.443398:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765778/real 1775765778]  req@ffff9f14a04d2300 x1862022114371968/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765803 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765778.443405:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d2300 x1862022114371968/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765803 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765778.443410:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765778.443411:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765778.443413:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114371968:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765778.443416:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114372096:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765778.443425:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765778.443431:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765778/real 1775765778]  req@ffff9f14a04d3800 x1862022114372096/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765803 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765778.443438:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3800 x1862022114372096/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765803 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765778.443442:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765778.443444:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765778.443446:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114372096:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765778.443449:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114372224:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765778.443457:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765778.443462:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765778/real 1775765778]  req@ffff9f14a04d0e00 x1862022114372224/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765803 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765778.443469:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0e00 x1862022114372224/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765803 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765778.443473:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765778.443475:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765778.443477:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114372224:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775765778.698707:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765779.466671:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775765779.722712:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765780.490647:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775765780.746773:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765781.514665:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775765781.770871:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765782.538633:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:2.0:1775765783.562644:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765783.562653:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1775765783.562656:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765783.562665:0:73045:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3161
00000100:00080000:2.0:1775765783.562674:0:73045:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765783.562682:0:73045:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775765783.562716:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3800 x1862022114372352/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775765783.562744:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765783.562747:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775765783.562748:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765783.562751:0:73045:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3161
00000100:00080000:2.0:1775765783.562753:0:73045:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765783.562755:0:73045:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775765783.562763:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775765783.562773:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0380 x1862022114372480/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775765783.562779:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765783.562783:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775765783.562784:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765783.562787:0:73045:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3161
00000100:00080000:2.0:1775765783.562789:0:73045:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00100000:1.0:1775765783.562793:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114372352:192.168.202.110@tcp:250:kworker.0
00000100:00080000:2.0:1775765783.562794:0:73045:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775765783.562801:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775765783.562807:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0000 x1862022114372608/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775765783.562814:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765783.562817:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775765783.562818:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765783.562820:0:73045:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3161
00000100:00080000:2.0:1775765783.562823:0:73045:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765783.562826:0:73045:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775765783.562829:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775765783.562837:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3b80 x1862022114372736/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775765783.562843:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765783.562845:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1775765783.562847:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765783.562850:0:73045:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3161
00000800:00000100:1.0:1775765783.562855:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:2.0:1775765783.562856:0:73045:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765783.562861:0:73045:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775765783.562864:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775765783.562871:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0e00 x1862022114372864/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775765783.562877:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765783.562879:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775765783.562881:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765783.562883:0:73045:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3161
00000100:00080000:2.0:1775765783.562886:0:73045:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765783.562888:0:73045:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775765783.562891:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775765783.562894:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765783/real 1775765783]  req@ffff9f14852d3800 x1862022114372352/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765808 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775765783.562901:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1880 x1862022114372992/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:1.0:1775765783.562915:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3800 x1862022114372352/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765808 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765783.562934:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765783.562935:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775765783.562941:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775765783.562944:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:2.0:1775765783.562945:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:1.0:1775765783.562949:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114372352:192.168.202.110@tcp:250:kworker.0
00000100:00080000:2.0:1775765783.562954:0:73045:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3161
00000100:00100000:1.0:1775765783.562958:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114372480:192.168.202.110@tcp:38:kworker.0
00000100:00080000:2.0:1775765783.562961:0:73045:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765783.562977:0:73045:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775765783.562982:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000800:00000100:1.0:1775765783.562991:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775765783.562995:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3100 x1862022114373120/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775765783.563003:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:1.0:1775765783.563004:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765783/real 1775765783]  req@ffff9f14852d0380 x1862022114372480/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765808 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765783.563009:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775765783.563013:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765783.563022:0:73045:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3161
00000100:00100000:1.0:1775765783.563025:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0380 x1862022114372480/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765808 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765783.563027:0:73045:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765783.563047:0:73045:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775765783.563055:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00080000:1.0:1775765783.563056:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765783.563062:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765783.563065:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114372480:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775765783.563068:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2300 x1862022114373248/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:1.0:1775765783.563102:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114372608:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765783.563126:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765783.563134:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765783/real 1775765783]  req@ffff9f14852d0000 x1862022114372608/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765808 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765783.563142:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0000 x1862022114372608/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765808 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765783.563147:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765783.563149:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765783.563151:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114372608:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765783.563156:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114372736:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765783.563167:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765783.563172:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765783/real 1775765783]  req@ffff9f14852d3b80 x1862022114372736/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765808 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765783.563180:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3b80 x1862022114372736/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765808 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765783.563185:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765783.563186:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765783.563188:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114372736:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775765783.563191:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114372864:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765783.563201:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765783.563206:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765783/real 1775765783]  req@ffff9f14852d0e00 x1862022114372864/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765808 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765783.563212:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0e00 x1862022114372864/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765808 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765783.563217:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765783.563219:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765783.563220:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114372864:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775765783.563223:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114372992:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765783.563233:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765783.563238:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765783/real 1775765783]  req@ffff9f14852d1880 x1862022114372992/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765808 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765783.563245:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1880 x1862022114372992/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765808 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765783.563250:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765783.563252:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765783.563253:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114372992:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765783.563257:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114373120:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765783.563265:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765783.563270:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765783/real 1775765783]  req@ffff9f14852d3100 x1862022114373120/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765808 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765783.563277:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3100 x1862022114373120/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765808 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765783.563282:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765783.563283:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765783.563285:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114373120:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765783.563288:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114373248:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765783.563297:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765783.563302:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765783/real 1775765783]  req@ffff9f14852d2300 x1862022114373248/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765808 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765783.563308:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2300 x1862022114373248/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765808 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765783.563317:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765783.563319:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765783.563321:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114373248:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775765783.818843:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765784.586682:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:2.0:1775765785.610684:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775765785.866716:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765786.634664:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775765786.890691:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765787.658662:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775765787.914738:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1775765788.682628:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765788.682636:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1775765788.682640:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765788.682648:0:73045:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3166
00000100:00080000:2.0:1775765788.682658:0:73045:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765788.682664:0:73045:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775765788.682702:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1c00 x1862022114373376/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775765788.682722:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765788.682724:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775765788.682726:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765788.682729:0:73045:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3166
00000100:00080000:2.0:1775765788.682731:0:73045:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765788.682733:0:73045:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775765788.682741:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775765788.682749:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3480 x1862022114373504/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775765788.682757:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765788.682759:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775765788.682760:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765788.682763:0:73045:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3166
00000100:00080000:2.0:1775765788.682765:0:73045:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765788.682767:0:73045:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775765788.682773:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114373376:192.168.202.110@tcp:250:kworker.0
00000008:00100000:2.0:1775765788.682773:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775765788.682784:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2300 x1862022114373632/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775765788.682791:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765788.682794:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775765788.682795:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765788.682798:0:73045:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3166
00000100:00080000:2.0:1775765788.682799:0:73045:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765788.682802:0:73045:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775765788.682804:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775765788.682812:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3100 x1862022114373760/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775765788.682817:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765788.682819:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1775765788.682821:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765788.682823:0:73045:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3166
00000100:00080000:2.0:1775765788.682826:0:73045:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765788.682827:0:73045:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775765788.682830:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:1.0:1775765788.682834:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775765788.682838:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1880 x1862022114373888/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775765788.682844:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765788.682846:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775765788.682848:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765788.682851:0:73045:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3166
00000100:00080000:2.0:1775765788.682854:0:73045:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765788.682856:0:73045:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775765788.682859:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775765788.682864:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0e00 x1862022114374016/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775765788.682870:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765788.682873:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775765788.682874:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765788.682877:0:73045:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3166
00000100:00080000:2.0:1775765788.682879:0:73045:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:1.0:1775765788.682882:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765788/real 1775765788]  req@ffff9f14852d1c00 x1862022114373376/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765813 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765788.682885:0:73045:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775765788.682910:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775765788.682923:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1c00 x1862022114373376/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765813 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775765788.682928:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3b80 x1862022114374144/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765788.682942:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765788.682968:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:2.0:1775765788.682969:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:1.0:1775765788.682973:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114373376:192.168.202.110@tcp:250:kworker.0
00000100:00080000:2.0:1775765788.682977:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00100000:1.0:1775765788.682997:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114373504:192.168.202.110@tcp:38:kworker.0
00000100:00080000:2.0:1775765788.683003:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765788.683009:0:73045:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3166
00000800:00000100:1.0:1775765788.683019:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:2.0:1775765788.683020:0:73045:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765788.683028:0:73045:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775765788.683032:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765788/real 1775765788]  req@ffff9f14852d3480 x1862022114373504/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765813 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:2.0:1775765788.683034:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775765788.683047:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3480 x1862022114373504/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765813 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775765788.683051:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0000 x1862022114374272/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765788.683065:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765788.683072:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765788.683076:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114373504:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765788.683097:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114373632:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765788.683108:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765788.683114:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765788/real 1775765788]  req@ffff9f14852d2300 x1862022114373632/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765813 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765788.683122:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2300 x1862022114373632/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765813 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765788.683127:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765788.683129:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765788.683131:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114373632:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765788.683135:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114373760:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765788.683144:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765788.683150:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765788/real 1775765788]  req@ffff9f14852d3100 x1862022114373760/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765813 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765788.683157:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3100 x1862022114373760/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765813 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765788.683162:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765788.683164:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765788.683165:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114373760:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775765788.683168:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114373888:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765788.683177:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765788.683182:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765788/real 1775765788]  req@ffff9f14852d1880 x1862022114373888/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765813 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765788.683190:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1880 x1862022114373888/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765813 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765788.683195:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765788.683196:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765788.683198:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114373888:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775765788.683201:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114374016:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765788.683210:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765788.683215:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765788/real 1775765788]  req@ffff9f14852d0e00 x1862022114374016/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765813 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765788.683222:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0e00 x1862022114374016/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765813 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765788.683227:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765788.683229:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765788.683230:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114374016:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765788.683233:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114374144:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765788.683242:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765788.683246:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765788/real 1775765788]  req@ffff9f14852d3b80 x1862022114374144/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765813 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765788.683253:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3b80 x1862022114374144/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765813 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765788.683263:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765788.683265:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765788.683267:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114374144:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765788.683270:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114374272:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765788.683278:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765788.683283:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765788/real 1775765788]  req@ffff9f14852d0000 x1862022114374272/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765813 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765788.683289:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0000 x1862022114374272/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765813 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765788.683294:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765788.683296:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765788.683298:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114374272:192.168.202.110@tcp:8:kworker.0
00010000:00010000:1.0:1775765788.938692:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00010000:00010000:2.0:1775765790.986733:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00010000:00010000:2.0:1775765793.034675:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1775765793.802618:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765793.802655:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1775765793.802660:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765793.802671:0:8:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3171
00000100:00080000:2.0:1775765793.802684:0:8:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765793.802693:0:8:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775765793.802737:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0380 x1862022114374400/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775765793.802771:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765793.802777:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775765793.802780:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765793.802793:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3171
00000100:00080000:2.0:1775765793.802799:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765793.802805:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775765793.802818:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775765793.802851:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3800 x1862022114374528/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775765793.802861:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:1.0:1775765793.802865:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114374400:192.168.202.110@tcp:250:kworker.0
00000100:00080000:2.0:1775765793.802865:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775765793.802866:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765793.802870:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3171
00000100:00080000:2.0:1775765793.802874:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765793.802877:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775765793.802882:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775765793.802890:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2d80 x1862022114374656/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775765793.802898:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765793.802901:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775765793.802902:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765793.802906:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3171
00000100:00080000:2.0:1775765793.802909:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765793.802912:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775765793.802916:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:1.0:1775765793.802922:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775765793.802924:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0700 x1862022114374784/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775765793.802948:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765793.802952:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1775765793.802953:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765793.802957:0:8:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3171
00000100:00080000:2.0:1775765793.802972:0:8:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765793.802975:0:8:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775765793.802980:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775765793.802990:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765793/real 1775765793]  req@ffff9f14852d0380 x1862022114374400/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765818 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775765793.802990:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0a80 x1862022114374912/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775765793.803001:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765793.803004:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775765793.803005:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:1.0:1775765793.803008:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0380 x1862022114374400/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765818 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765793.803009:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3171
00000100:00080000:2.0:1775765793.803013:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765793.803017:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00080000:1.0:1775765793.803019:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000008:00100000:2.0:1775765793.803021:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00080000:1.0:1775765793.803023:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765793.803028:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114374400:192.168.202.110@tcp:250:kworker.0
00000100:00100000:2.0:1775765793.803029:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1500 x1862022114375040/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775765793.803038:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765793.803040:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775765793.803042:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765793.803045:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3171
00000100:00080000:2.0:1775765793.803049:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:1.0:1775765793.803050:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114374528:192.168.202.110@tcp:38:kworker.0
00000100:00080000:2.0:1775765793.803052:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775765793.803056:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775765793.803065:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2680 x1862022114375168/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:1.0:1775765793.803072:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:2.0:1775765793.803075:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765793.803078:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775765793.803080:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:1.0:1775765793.803084:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765793/real 1775765793]  req@ffff9f14852d3800 x1862022114374528/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765818 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765793.803084:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3171
00000100:00080000:2.0:1775765793.803088:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765793.803091:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775765793.803096:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775765793.803097:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3800 x1862022114374528/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765818 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765793.803105:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00100000:2.0:1775765793.803106:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2a00 x1862022114375296/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765793.803109:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765793.803113:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114374528:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765793.803119:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114374656:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765793.803144:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765793.803155:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765793/real 1775765793]  req@ffff9f14852d2d80 x1862022114374656/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765818 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765793.803167:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2d80 x1862022114374656/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765818 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765793.803172:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765793.803175:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765793.803177:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114374656:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765793.803181:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114374784:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765793.803202:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765793.803226:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765793/real 1775765793]  req@ffff9f14852d0700 x1862022114374784/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765818 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765793.803255:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0700 x1862022114374784/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765818 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765793.803268:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765793.803273:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765793.803278:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114374784:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775765793.803285:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114374912:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765793.803342:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765793.803354:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765793/real 1775765793]  req@ffff9f14852d0a80 x1862022114374912/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765818 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765793.803367:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0a80 x1862022114374912/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765818 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765793.803376:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765793.803379:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765793.803383:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114374912:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775765793.803389:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114375040:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765793.803405:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765793.803413:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765793/real 1775765793]  req@ffff9f14852d1500 x1862022114375040/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765818 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765793.803420:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1500 x1862022114375040/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765818 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765793.803426:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765793.803428:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765793.803430:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114375040:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765793.803444:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114375168:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765793.803454:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765793.803459:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765793/real 1775765793]  req@ffff9f14852d2680 x1862022114375168/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765818 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765793.803467:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2680 x1862022114375168/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765818 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765793.803472:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765793.803473:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765793.803475:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114375168:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765793.803479:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114375296:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765793.803488:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765793.803493:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765793/real 1775765793]  req@ffff9f14852d2a00 x1862022114375296/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765818 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765793.803501:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2a00 x1862022114375296/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765818 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765793.803505:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765793.803507:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765793.803509:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114375296:192.168.202.110@tcp:8:kworker.0
00010000:00010000:3.0:1775765794.058904:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00010000:00010000:3.0:1775765795.082774:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00010000:00010000:1.0:1775765796.106683:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00010000:00010000:2.0:1775765798.154729:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1775765798.922668:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:2.0:1775765798.922673:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775765798.922681:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775765798.922685:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765798.922695:0:8:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3176
00000100:00080000:3.0:1775765798.922705:0:8:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765798.922712:0:8:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775765798.922748:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745500 x1862022114375424/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765798.922787:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765798.922792:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775765798.922793:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765798.922798:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3176
00000100:00080000:3.0:1775765798.922803:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765798.922808:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765798.922819:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775765798.922820:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114375424:192.168.202.110@tcp:250:kworker.0
00000100:00100000:3.0:1775765798.922840:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747480 x1862022114375552/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765798.922853:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765798.922857:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775765798.922859:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765798.922864:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3176
00000100:00080000:3.0:1775765798.922869:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765798.922873:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765798.922880:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775765798.922908:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744700 x1862022114375680/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765798.922919:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765798.922940:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775765798.922943:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:2.0:1775765798.922950:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114375552:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775765798.922950:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3176
00000100:00080000:3.0:1775765798.922959:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765798.922964:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765798.922970:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775765798.922983:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745f80 x1862022114375808/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775765798.922994:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114375680:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775765798.922995:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765798.923003:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775765798.923004:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765798.923010:0:8:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3176
00000100:00080000:3.0:1775765798.923014:0:8:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765798.923019:0:8:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765798.923025:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775765798.923035:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114375808:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775765798.924003:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744000 x1862022114375936/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765798.924028:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765798.924032:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775765798.924034:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765798.924038:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3176
00000100:00080000:3.0:1775765798.924044:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765798.924049:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765798.924056:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775765798.924067:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747800 x1862022114376064/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765798.924076:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765798.924080:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775765798.924081:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765798.924085:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3176
00000100:00080000:3.0:1775765798.924088:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765798.924091:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765798.924096:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775765798.924106:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745c00 x1862022114376192/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765798.924115:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765798.924119:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775765798.924121:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765798.924138:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3176
00000100:00080000:3.0:1775765798.924143:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765798.924148:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765798.924153:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775765798.924164:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745180 x1862022114376320/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775765798.924207:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114375936:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775765798.924255:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114376064:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775765798.924278:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114376192:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775765798.924300:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114376320:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775765799.946622:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775765800.202711:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765800.970743:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775765801.226855:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765801.994619:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000400:00000100:1.0:1775765801.994759:0:2393:0:(lib-socket.c:454:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.202.110:988
00000400:00000100:1.0:1775765801.994795:0:2393:0:(acceptor.c:111:lnet_connect_console_error()) Connection to 192.168.202.110@tcp at host 192.168.202.110 was unreachable: the network or that node may be down, or Lustre may be misconfigured.
00000800:00000100:1.0:1775765801.994808:0:2393:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:1.0:1775765801.994839:0:2393:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:1.0:1775765801.994845:0:2393:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:1.0:1775765801.994879:0:2393:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000100:00100000:2.0:1775765801.994885:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765798/real 1775765801]  req@ffff9f1485747480 x1862022114375552/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765823 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000800:00000100:1.0:1775765801.994890:0:2393:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:1.0:1775765801.994901:0:2393:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:1.0:1775765801.994911:0:2393:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:1.0:1775765801.994917:0:2393:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000100:00100000:2.0:1775765801.994928:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747480 x1862022114375552/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765823 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765801.994937:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775765801.994941:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775765801.994944:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114375552:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775765801.994949:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765798/real 1775765801]  req@ffff9f1485744700 x1862022114375680/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765823 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775765801.994958:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744700 x1862022114375680/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765823 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765801.994963:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775765801.994965:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775765801.994967:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114375680:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775765801.994972:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765798/real 1775765801]  req@ffff9f1485745500 x1862022114375424/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765823 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775765801.994979:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745500 x1862022114375424/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765823 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765801.994984:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775765801.994986:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775765801.994988:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114375424:192.168.202.110@tcp:250:kworker.0
00000100:00100000:2.0:1775765801.995007:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765798/real 1775765801]  req@ffff9f1485744000 x1862022114375936/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765823 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775765801.995016:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744000 x1862022114375936/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765823 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765801.995021:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775765801.995022:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775765801.995024:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114375936:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775765801.995028:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765798/real 1775765801]  req@ffff9f1485747800 x1862022114376064/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765823 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775765801.995035:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747800 x1862022114376064/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765823 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765801.995039:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775765801.995041:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775765801.995043:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114376064:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775765801.995047:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765798/real 1775765801]  req@ffff9f1485745c00 x1862022114376192/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765823 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775765801.995054:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745c00 x1862022114376192/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765823 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765801.995059:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775765801.995061:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775765801.995063:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114376192:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775765801.995066:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765798/real 1775765801]  req@ffff9f1485745180 x1862022114376320/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765823 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775765801.995074:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745180 x1862022114376320/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765823 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765801.995079:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775765801.995081:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775765801.995083:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114376320:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775765801.995087:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765798/real 1775765801]  req@ffff9f1485745f80 x1862022114375808/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765823 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775765801.995093:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745f80 x1862022114375808/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765823 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765801.995097:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775765801.995099:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775765801.995101:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114375808:192.168.202.110@tcp:8:kworker.0
00010000:00010000:3.0:1775765802.250705:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765803.018648:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775765803.274695:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1775765804.042644:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765804.042652:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1775765804.042657:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765804.042667:0:73045:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3181
00000100:00080000:2.0:1775765804.042677:0:73045:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765804.042683:0:73045:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775765804.042717:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1f80 x1862022114376448/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775765804.042748:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765804.042752:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775765804.042753:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765804.042757:0:73045:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3181
00000100:00080000:2.0:1775765804.042760:0:73045:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765804.042764:0:73045:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775765804.042773:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775765804.042787:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1180 x1862022114376576/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:1.0:1775765804.042788:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114376448:192.168.202.110@tcp:250:kworker.0
00000100:00080000:2.0:1775765804.042811:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765804.042814:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775765804.042815:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765804.042819:0:73045:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3181
00000100:00080000:2.0:1775765804.042823:0:73045:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765804.042827:0:73045:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775765804.042833:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775765804.042862:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2a00 x1862022114376704/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775765804.042872:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765804.042876:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775765804.042879:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765804.042884:0:73045:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3181
00000800:00000100:1.0:1775765804.042890:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:2.0:1775765804.042894:0:73045:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765804.042902:0:73045:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775765804.042911:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775765804.042927:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2680 x1862022114376832/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:1.0:1775765804.042938:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765804/real 1775765804]  req@ffff9f14852d1f80 x1862022114376448/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765829 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765804.042940:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765804.042949:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1775765804.042951:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765804.042962:0:73045:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3181
00000100:00080000:2.0:1775765804.042969:0:73045:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765804.042973:0:73045:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775765804.042978:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775765804.042984:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1f80 x1862022114376448/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765829 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775765804.042990:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1500 x1862022114376960/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775765804.043020:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765804.043023:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775765804.043023:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775765804.043027:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765804.043036:0:73045:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3181
00000100:00080000:1.0:1775765804.043037:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:2.0:1775765804.043039:0:73045:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765804.043048:0:73045:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775765804.043050:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114376448:192.168.202.110@tcp:250:kworker.0
00000008:00100000:2.0:1775765804.043052:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775765804.043064:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0a80 x1862022114377088/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775765804.043079:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765804.043081:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775765804.043084:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765804.043087:0:73045:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3181
00000100:00080000:2.0:1775765804.043089:0:73045:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765804.043093:0:73045:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775765804.043097:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775765804.043099:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114376576:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775765804.043107:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0700 x1862022114377216/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775765804.043116:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765804.043119:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775765804.043120:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765804.043123:0:73045:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3181
00000100:00080000:2.0:1775765804.043127:0:73045:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765804.043131:0:73045:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775765804.043138:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:1.0:1775765804.043144:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775765804.043157:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2d80 x1862022114377344/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:1.0:1775765804.043168:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765804/real 1775765804]  req@ffff9f14852d1180 x1862022114376576/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765829 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765804.043186:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1180 x1862022114376576/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765829 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765804.043197:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765804.043201:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765804.043205:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114376576:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765804.043213:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114376704:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765804.043260:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765804.043274:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765804/real 1775765804]  req@ffff9f14852d2a00 x1862022114376704/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765829 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765804.043296:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2a00 x1862022114376704/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765829 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765804.043305:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765804.043309:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765804.043313:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114376704:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765804.043321:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114376832:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765804.043345:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765804.043368:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765804/real 1775765804]  req@ffff9f14852d2680 x1862022114376832/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765829 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765804.043381:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2680 x1862022114376832/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765829 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765804.043390:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765804.043394:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765804.043398:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114376832:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775765804.043404:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114376960:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765804.043424:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765804.043435:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765804/real 1775765804]  req@ffff9f14852d1500 x1862022114376960/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765829 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765804.043447:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1500 x1862022114376960/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765829 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765804.043455:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765804.043458:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765804.043462:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114376960:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775765804.043468:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114377088:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765804.043489:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765804.043501:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765804/real 1775765804]  req@ffff9f14852d0a80 x1862022114377088/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765829 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765804.043513:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0a80 x1862022114377088/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765829 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765804.043523:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765804.043527:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765804.043532:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114377088:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765804.043584:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114377216:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765804.043611:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765804.043625:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765804/real 1775765804]  req@ffff9f14852d0700 x1862022114377216/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765829 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765804.043639:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0700 x1862022114377216/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765829 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765804.043647:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765804.043651:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765804.043655:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114377216:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765804.043662:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114377344:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765804.043699:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765804.043711:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765804/real 1775765804]  req@ffff9f14852d2d80 x1862022114377344/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765829 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765804.043724:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2d80 x1862022114377344/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765829 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765804.043732:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765804.043735:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765804.043738:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114377344:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775765805.322909:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00010000:00010000:2.0:1775765807.370848:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00010000:00010000:3.0:1775765808.394879:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1775765809.162703:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765809.162721:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1775765809.162727:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765809.162739:0:8:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3186
00000100:00080000:2.0:1775765809.162754:0:8:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765809.162764:0:8:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775765809.162861:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3800 x1862022114377472/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775765809.162897:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765809.162901:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775765809.162903:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765809.162908:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3186
00000100:00080000:2.0:1775765809.162913:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765809.162919:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775765809.162931:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775765809.162943:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114377472:192.168.202.110@tcp:250:kworker.0
00000100:00100000:2.0:1775765809.162946:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0380 x1862022114377600/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775765809.162968:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765809.162972:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775765809.162974:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765809.162981:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3186
00000100:00080000:2.0:1775765809.162985:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765809.162988:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775765809.162993:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775765809.163004:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0000 x1862022114377728/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775765809.163012:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765809.163015:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775765809.163017:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765809.163025:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3186
00000800:00000100:1.0:1775765809.163026:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:2.0:1775765809.163038:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765809.163045:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775765809.163051:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775765809.163065:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3b80 x1862022114377856/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775765809.163075:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:1.0:1775765809.163080:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765809/real 1775765809]  req@ffff9f14852d3800 x1862022114377472/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765834 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765809.163081:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1775765809.163087:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765809.163106:0:8:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3186
00000100:00080000:2.0:1775765809.163118:0:8:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765809.163124:0:8:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775765809.163129:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775765809.163132:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3800 x1862022114377472/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765834 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765809.163147:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00100000:2.0:1775765809.163149:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0e00 x1862022114377984/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765809.163159:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765809.163178:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114377472:192.168.202.110@tcp:250:kworker.0
00000100:00080000:2.0:1775765809.163192:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765809.163196:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775765809.163199:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765809.163205:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3186
00000100:00080000:2.0:1775765809.163208:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765809.163215:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775765809.163218:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114377600:192.168.202.110@tcp:38:kworker.0
00000008:00100000:2.0:1775765809.163222:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775765809.163249:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1880 x1862022114378112/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775765809.163262:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000800:00000100:1.0:1775765809.163272:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:2.0:1775765809.163272:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775765809.163277:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765809.163283:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3186
00000100:00080000:2.0:1775765809.163289:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765809.163293:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775765809.163297:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775765809.163301:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765809/real 1775765809]  req@ffff9f14852d0380 x1862022114377600/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765834 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775765809.163311:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3100 x1862022114378240/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:1.0:1775765809.163322:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0380 x1862022114377600/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765834 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765809.163338:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765809.163355:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775765809.163358:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775765809.163358:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765809.163373:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3186
00000100:00080000:1.0:1775765809.163374:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00080000:2.0:1775765809.163382:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:1.0:1775765809.163386:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114377600:192.168.202.110@tcp:38:kworker.0
00000100:00080000:2.0:1775765809.163391:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775765809.163407:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775765809.163414:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114377728:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775765809.163427:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2300 x1862022114378368/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:1.0:1775765809.163442:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765809.163459:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765809/real 1775765809]  req@ffff9f14852d0000 x1862022114377728/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765834 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765809.163474:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0000 x1862022114377728/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765834 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765809.163485:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765809.163488:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765809.163494:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114377728:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765809.163500:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114377856:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765809.163511:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765809.163516:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765809/real 1775765809]  req@ffff9f14852d3b80 x1862022114377856/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765834 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765809.163523:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3b80 x1862022114377856/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765834 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765809.163528:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765809.163530:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765809.163532:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114377856:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775765809.163536:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114377984:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765809.163545:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765809.163551:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765809/real 1775765809]  req@ffff9f14852d0e00 x1862022114377984/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765834 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765809.163591:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0e00 x1862022114377984/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765834 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765809.163596:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765809.163599:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765809.163601:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114377984:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775765809.163615:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114378112:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765809.163629:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765809.163635:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765809/real 1775765809]  req@ffff9f14852d1880 x1862022114378112/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765834 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765809.163643:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1880 x1862022114378112/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765834 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765809.163648:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765809.163650:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765809.163652:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114378112:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765809.163656:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114378240:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765809.163665:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765809.163671:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765809/real 1775765809]  req@ffff9f14852d3100 x1862022114378240/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765834 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765809.163678:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3100 x1862022114378240/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765834 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765809.163683:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765809.163684:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765809.163686:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114378240:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765809.163689:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114378368:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765809.163699:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765809.163704:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765809/real 1775765809]  req@ffff9f14852d2300 x1862022114378368/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765834 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765809.163711:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2300 x1862022114378368/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765834 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765809.163716:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765809.163718:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765809.163720:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114378368:192.168.202.110@tcp:8:kworker.0
00010000:00010000:3.0:1775765809.418754:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00010000:00010000:2.0:1775765810.442816:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00010000:00010000:2.0:1775765812.490729:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765813.258745:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:2.0:1775765814.282647:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775765814.282687:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765814.282696:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775765814.282701:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765814.282710:0:8:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3191
00000100:00080000:3.0:1775765814.282721:0:8:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765814.282728:0:8:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775765814.282792:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747100 x1862022114378496/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765814.282819:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765814.282822:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775765814.282824:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765814.282828:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3191
00000100:00080000:3.0:1775765814.282831:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765814.282834:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765814.282843:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775765814.282856:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745f80 x1862022114378624/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765814.282866:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765814.282872:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00100000:1.0:1775765814.282873:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114378496:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775765814.282874:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765814.282883:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3191
00000100:00080000:3.0:1775765814.282886:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765814.282889:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765814.282893:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775765814.282905:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745180 x1862022114378752/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765814.282914:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765814.282917:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775765814.282919:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765814.282922:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3191
00000100:00080000:3.0:1775765814.282925:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765814.282928:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765814.282932:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775765814.282941:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745c00 x1862022114378880/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:1.0:1775765814.282946:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775765814.282957:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765814.282960:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775765814.282962:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765814.282967:0:8:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3191
00000100:00080000:3.0:1775765814.282971:0:8:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765814.282974:0:8:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765814.282978:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775765814.282986:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747800 x1862022114379008/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765814.282994:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:1.0:1775765814.282995:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765814/real 1775765814]  req@ffff9f1485747100 x1862022114378496/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765839 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775765814.283005:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775765814.283009:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765814.283019:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3191
00000100:00080000:3.0:1775765814.283023:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765814.283027:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775765814.283031:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747100 x1862022114378496/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765839 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:3.0:1775765814.283034:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00080000:1.0:1775765814.283048:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00100000:3.0:1775765814.283050:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744000 x1862022114379136/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765814.283056:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765814.283066:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114378496:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775765814.283070:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765814.283074:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775765814.283076:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765814.283080:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3191
00000100:00080000:3.0:1775765814.283084:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765814.283086:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765814.283091:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775765814.283095:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114378624:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775765814.283101:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745500 x1862022114379264/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765814.283113:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765814.283116:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775765814.283118:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000800:00000100:1.0:1775765814.283123:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775765814.283127:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3191
00000100:00080000:3.0:1775765814.283133:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765814.283136:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765814.283140:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775765814.283142:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765814/real 1775765814]  req@ffff9f1485745f80 x1862022114378624/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765839 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775765814.283153:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744700 x1862022114379392/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:1.0:1775765814.283160:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745f80 x1862022114378624/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765839 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765814.283210:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765814.283214:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765814.283218:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114378624:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765814.283228:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114378752:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765814.283256:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765814.283270:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765814/real 1775765814]  req@ffff9f1485745180 x1862022114378752/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765839 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765814.283285:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745180 x1862022114378752/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765839 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765814.283297:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765814.283301:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765814.283304:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114378752:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765814.283310:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114378880:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765814.283330:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765814.283339:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765814/real 1775765814]  req@ffff9f1485745c00 x1862022114378880/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765839 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765814.283349:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745c00 x1862022114378880/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765839 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765814.283356:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765814.283358:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765814.283362:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114378880:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775765814.283366:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114379008:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765814.283379:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765814.283386:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765814/real 1775765814]  req@ffff9f1485747800 x1862022114379008/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765839 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765814.283395:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747800 x1862022114379008/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765839 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765814.283401:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765814.283403:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765814.283406:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114379008:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775765814.283410:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114379136:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765814.283422:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765814.283428:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765814/real 1775765814]  req@ffff9f1485744000 x1862022114379136/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765839 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765814.283437:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744000 x1862022114379136/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765839 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765814.283443:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765814.283446:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765814.283448:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114379136:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765814.283463:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114379264:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765814.283476:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765814.283484:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765814/real 1775765814]  req@ffff9f1485745500 x1862022114379264/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765839 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765814.283493:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745500 x1862022114379264/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765839 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765814.283500:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765814.283502:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765814.283504:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114379264:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765814.283508:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114379392:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765814.283520:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765814.283527:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765814/real 1775765814]  req@ffff9f1485744700 x1862022114379392/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765839 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765814.283536:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744700 x1862022114379392/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765839 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765814.283542:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765814.283544:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765814.283547:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114379392:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775765814.538904:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765815.306622:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775765815.562776:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765816.330637:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775765816.586738:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765817.354664:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775765817.610780:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765818.378755:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:2.0:1775765819.402667:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775765819.402779:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765819.402790:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775765819.402795:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765819.402807:0:8:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3196
00000100:00080000:3.0:1775765819.402820:0:8:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765819.402828:0:8:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775765819.402871:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747480 x1862022114379520/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765819.402901:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765819.402905:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775765819.402907:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765819.402911:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3196
00000100:00080000:3.0:1775765819.402915:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765819.402918:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765819.402927:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775765819.402939:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744380 x1862022114379648/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765819.402950:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765819.402953:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775765819.402955:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765819.402958:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3196
00000100:00080000:3.0:1775765819.402964:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00100000:1.0:1775765819.402967:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114379520:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775765819.402971:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765819.402982:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775765819.402994:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746a00 x1862022114379776/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765819.403022:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765819.403026:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775765819.403029:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765819.403033:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3196
00000100:00080000:3.0:1775765819.403039:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765819.403043:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765819.403049:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775765819.403066:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746300 x1862022114379904/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:1.0:1775765819.403073:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775765819.403085:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765819.403106:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775765819.403107:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765819.403111:0:8:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3196
00000100:00080000:3.0:1775765819.403115:0:8:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765819.403119:0:8:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765819.403123:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775765819.403134:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747b80 x1862022114380032/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765819.403149:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765819.403153:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775765819.403157:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:1.0:1775765819.403158:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765819/real 1775765819]  req@ffff9f1485747480 x1862022114379520/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765844 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775765819.403164:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3196
00000100:00080000:3.0:1775765819.403169:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765819.403178:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765819.403185:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775765819.403194:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747480 x1862022114379520/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765844 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775765819.403201:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744e00 x1862022114380160/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765819.403226:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775765819.403227:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765819.403234:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775765819.403237:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:3.0:1775765819.403238:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:1.0:1775765819.403249:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114379520:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775765819.403250:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3196
00000100:00080000:3.0:1775765819.403257:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765819.403262:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775765819.403266:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114379648:192.168.202.110@tcp:38:kworker.0
00000008:00100000:3.0:1775765819.403269:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775765819.403285:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746680 x1862022114380288/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765819.403294:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000800:00000100:1.0:1775765819.403301:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775765819.403301:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775765819.403305:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765819.403312:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3196
00000100:00080000:3.0:1775765819.403318:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:1.0:1775765819.403322:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765819/real 1775765819]  req@ffff9f1485744380 x1862022114379648/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765844 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775765819.403326:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765819.403335:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775765819.403352:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744380 x1862022114379648/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765844 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775765819.403356:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745880 x1862022114380416/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765819.403385:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765819.403395:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765819.403400:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114379648:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765819.403425:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114379776:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765819.403441:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765819.403449:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765819/real 1775765819]  req@ffff9f1485746a00 x1862022114379776/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765844 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765819.403459:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746a00 x1862022114379776/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765844 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765819.403466:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765819.403469:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765819.403472:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114379776:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765819.403478:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114379904:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765819.403507:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765819.403518:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765819/real 1775765819]  req@ffff9f1485746300 x1862022114379904/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765844 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765819.403530:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746300 x1862022114379904/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765844 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765819.403538:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765819.403541:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765819.403544:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114379904:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775765819.403549:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114380032:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765819.403564:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765819.403614:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765819/real 1775765819]  req@ffff9f1485747b80 x1862022114380032/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765844 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765819.403626:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747b80 x1862022114380032/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765844 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765819.403634:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765819.403638:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765819.403641:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114380032:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775765819.403647:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114380160:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765819.403668:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765819.403678:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765819/real 1775765819]  req@ffff9f1485744e00 x1862022114380160/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765844 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765819.403737:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744e00 x1862022114380160/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765844 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765819.403745:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765819.403747:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765819.403750:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114380160:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765819.403755:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114380288:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765819.403774:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765819.403783:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765819/real 1775765819]  req@ffff9f1485746680 x1862022114380288/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765844 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765819.403796:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746680 x1862022114380288/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765844 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765819.403807:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765819.403811:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765819.403816:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114380288:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765819.403823:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114380416:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765819.403849:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765819.403864:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765819/real 1775765819]  req@ffff9f1485745880 x1862022114380416/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765844 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765819.403878:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745880 x1862022114380416/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765844 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765819.403891:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765819.403894:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765819.403896:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114380416:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775765819.658781:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765820.426697:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:2.0:1775765821.450735:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775765821.706745:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765822.474637:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775765822.730781:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765823.498664:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775765823.754755:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1775765824.522751:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765824.522760:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1775765824.522766:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765824.522774:0:73045:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3202
00000100:00080000:2.0:1775765824.522784:0:73045:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765824.522789:0:73045:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775765824.522839:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3480 x1862022114380544/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775765824.522868:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765824.522872:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775765824.522875:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765824.522879:0:73045:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3202
00000100:00080000:2.0:1775765824.522884:0:73045:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765824.522889:0:73045:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775765824.522910:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775765824.522925:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114380544:192.168.202.110@tcp:250:kworker.0
00000100:00100000:2.0:1775765824.522925:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1c00 x1862022114380672/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775765824.522940:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765824.522942:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775765824.522944:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765824.522947:0:73045:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3202
00000100:00080000:2.0:1775765824.522950:0:73045:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765824.522953:0:73045:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775765824.522957:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775765824.522967:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2300 x1862022114380800/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775765824.522974:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765824.523003:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775765824.523005:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765824.523008:0:73045:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3202
00000100:00080000:2.0:1775765824.523013:0:73045:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000800:00000100:1.0:1775765824.523019:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:2.0:1775765824.523020:0:73045:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775765824.523030:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775765824.523043:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3100 x1862022114380928/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775765824.523063:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765824.523066:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1775765824.523069:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765824.523075:0:73045:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3202
00000100:00080000:2.0:1775765824.523080:0:73045:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00100000:1.0:1775765824.523085:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765824/real 1775765824]  req@ffff9f14852d3480 x1862022114380544/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765849 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765824.523089:0:73045:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775765824.523099:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775765824.523114:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1880 x1862022114381056/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:1.0:1775765824.523119:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3480 x1862022114380544/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765849 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765824.523148:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765824.523154:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775765824.523156:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775765824.523157:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765824.523171:0:73045:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3202
00000100:00080000:1.0:1775765824.523173:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:2.0:1775765824.523177:0:73045:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:1.0:1775765824.523182:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114380544:192.168.202.110@tcp:250:kworker.0
00000100:00080000:2.0:1775765824.523189:0:73045:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775765824.523202:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775765824.523213:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0e00 x1862022114381184/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775765824.523225:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:1.0:1775765824.523241:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114380672:192.168.202.110@tcp:38:kworker.0
00000100:00080000:2.0:1775765824.523248:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775765824.523251:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765824.523256:0:73045:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3202
00000100:00080000:2.0:1775765824.523259:0:73045:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765824.523263:0:73045:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775765824.523268:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000800:00000100:1.0:1775765824.523278:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775765824.523281:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3b80 x1862022114381312/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775765824.523291:0:73045:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765824.523294:0:73045:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775765824.523296:0:73045:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765824.523301:0:73045:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3202
00000100:00100000:1.0:1775765824.523302:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765824/real 1775765824]  req@ffff9f14852d1c00 x1862022114380672/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765849 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765824.523303:0:73045:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765824.523310:0:73045:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775765824.523334:0:73045:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775765824.523352:0:73045:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0000 x1862022114381440/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:1.0:1775765824.523359:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1c00 x1862022114380672/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765849 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765824.523376:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765824.523381:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765824.523385:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114380672:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765824.523395:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114380800:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765824.523422:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765824.523433:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765824/real 1775765824]  req@ffff9f14852d2300 x1862022114380800/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765849 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765824.523445:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2300 x1862022114380800/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765849 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765824.523453:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765824.523457:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765824.523460:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114380800:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765824.523465:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114380928:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765824.523479:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765824.523487:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765824/real 1775765824]  req@ffff9f14852d3100 x1862022114380928/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765849 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765824.523498:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3100 x1862022114380928/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765849 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765824.523506:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765824.523509:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765824.523512:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114380928:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775765824.523517:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114381056:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765824.523530:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765824.523610:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765824/real 1775765824]  req@ffff9f14852d1880 x1862022114381056/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765849 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765824.523622:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1880 x1862022114381056/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765849 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765824.523629:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765824.523631:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765824.523634:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114381056:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775765824.523670:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114381184:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765824.523716:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765824.523724:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765824/real 1775765824]  req@ffff9f14852d0e00 x1862022114381184/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765849 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765824.523732:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0e00 x1862022114381184/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765849 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765824.523738:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765824.523740:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765824.523742:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114381184:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765824.523746:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114381312:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765824.523756:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765824.523762:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765824/real 1775765824]  req@ffff9f14852d3b80 x1862022114381312/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765849 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765824.523769:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3b80 x1862022114381312/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765849 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765824.523779:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765824.523781:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765824.523783:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114381312:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765824.523786:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114381440:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765824.523796:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765824.523816:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765824/real 1775765824]  req@ffff9f14852d0000 x1862022114381440/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765849 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765824.523824:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0000 x1862022114381440/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765849 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765824.523829:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765824.523831:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765824.523833:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114381440:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775765824.778780:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00010000:00010000:2.0:1775765826.826735:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00010000:00010000:2.0:1775765828.874762:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1775765829.642672:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765829.642679:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1775765829.642683:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765829.642693:0:8:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3207
00000100:00080000:2.0:1775765829.642705:0:8:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765829.642713:0:8:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775765829.642751:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0380 x1862022114381568/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775765829.642780:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765829.642783:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775765829.642785:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765829.642790:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3207
00000100:00080000:2.0:1775765829.642794:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765829.642798:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775765829.642809:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775765829.642823:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114381568:192.168.202.110@tcp:250:kworker.0
00000100:00100000:2.0:1775765829.642824:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3800 x1862022114381696/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775765829.642842:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765829.642846:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775765829.642848:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765829.642852:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3207
00000100:00080000:2.0:1775765829.642857:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765829.642861:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775765829.642866:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775765829.642877:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2d80 x1862022114381824/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775765829.642887:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000800:00000100:1.0:1775765829.642889:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:2.0:1775765829.642894:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775765829.642896:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765829.642901:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3207
00000100:00080000:2.0:1775765829.642906:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765829.642911:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775765829.642916:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775765829.642928:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765829/real 1775765829]  req@ffff9f14852d0380 x1862022114381568/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765854 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775765829.642932:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0700 x1862022114381952/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775765829.642949:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:1.0:1775765829.642952:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0380 x1862022114381568/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765854 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765829.642955:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1775765829.642959:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765829.642968:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775765829.642968:0:8:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3207
00000100:00080000:2.0:1775765829.642974:0:8:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765829.642976:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:2.0:1775765829.642981:0:8:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775765829.642982:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114381568:192.168.202.110@tcp:250:kworker.0
00000008:00100000:2.0:1775765829.642989:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775765829.643002:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0a80 x1862022114382080/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:1.0:1775765829.643006:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114381696:192.168.202.110@tcp:38:kworker.0
00000100:00080000:2.0:1775765829.643022:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765829.643026:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000800:00000100:1.0:1775765829.643031:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:2.0:1775765829.643032:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765829.643038:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3207
00000100:00080000:2.0:1775765829.643043:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:1.0:1775765829.643049:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765829/real 1775765829]  req@ffff9f14852d3800 x1862022114381696/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765854 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765829.643049:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775765829.643057:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775765829.643074:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1500 x1862022114382208/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:1.0:1775765829.643075:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3800 x1862022114381696/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765854 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765829.643104:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765829.643105:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765829.643112:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00080000:2.0:1775765829.643112:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775765829.643116:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:1.0:1775765829.643118:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114381696:192.168.202.110@tcp:38:kworker.0
00000100:00080000:2.0:1775765829.643127:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3207
00000100:00100000:1.0:1775765829.643128:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114381824:192.168.202.110@tcp:38:kworker.0
00000100:00080000:2.0:1775765829.643134:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765829.643141:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775765829.643147:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000800:00000100:1.0:1775765829.643151:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765829.643163:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765829/real 1775765829]  req@ffff9f14852d2d80 x1862022114381824/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765854 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775765829.643163:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2680 x1862022114382336/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:1.0:1775765829.643198:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2d80 x1862022114381824/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765854 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765829.643198:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765829.643207:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775765829.643211:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765829.643213:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765829.643219:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00080000:2.0:1775765829.643222:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3207
00000100:00100000:1.0:1775765829.643225:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114381824:192.168.202.110@tcp:38:kworker.0
00000100:00080000:2.0:1775765829.643228:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:1.0:1775765829.643231:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114381952:192.168.202.110@tcp:8:kworker.0
00000100:00080000:2.0:1775765829.643236:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775765829.643242:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:1.0:1775765829.643248:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775765829.643257:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2a00 x1862022114382464/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:1.0:1775765829.643258:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765829/real 1775765829]  req@ffff9f14852d0700 x1862022114381952/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765854 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765829.643289:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0700 x1862022114381952/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765854 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765829.643296:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765829.643298:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765829.643301:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114381952:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775765829.643315:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114382080:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765829.643325:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765829.643331:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765829/real 1775765829]  req@ffff9f14852d0a80 x1862022114382080/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765854 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765829.643338:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0a80 x1862022114382080/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765854 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765829.643343:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765829.643345:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765829.643347:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114382080:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775765829.643350:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114382208:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765829.643359:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765829.643365:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765829/real 1775765829]  req@ffff9f14852d1500 x1862022114382208/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765854 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765829.643372:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1500 x1862022114382208/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765854 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765829.643381:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765829.643383:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765829.643386:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114382208:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765829.643389:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114382336:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775765829.643397:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765829.643402:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765829/real 1775765829]  req@ffff9f14852d2680 x1862022114382336/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765854 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765829.643409:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2680 x1862022114382336/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765854 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765829.643414:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765829.643416:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765829.643417:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114382336:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765829.643420:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114382464:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775765829.643430:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765829.643434:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765829/real 1775765829]  req@ffff9f14852d2a00 x1862022114382464/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765854 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765829.643441:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2a00 x1862022114382464/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765854 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765829.643445:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765829.643447:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775765829.643450:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114382464:192.168.202.110@tcp:8:kworker.0
00010000:00010000:3.0:1775765829.898746:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00010000:00010000:3.0:1775765830.922753:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00010000:00010000:2.0:1775765831.946786:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00010000:00010000:2.0:1775765833.994666:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765834.762688:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:2.0:1775765834.762907:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114382592:192.168.202.110@tcp:250:kworker.0
00000800:00000100:2.0:1775765834.763016:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775765834.763081:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765834/real 1775765834]  req@ffff9f14a04d0a80 x1862022114382592/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765859 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775765834.763142:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0a80 x1862022114382592/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765859 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765834.763209:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775765834.763226:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775765834.763260:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114382592:192.168.202.110@tcp:250:kworker.0
00000100:00100000:2.0:1775765834.763300:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114382720:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775765834.763357:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775765834.763385:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765834/real 1775765834]  req@ffff9f14a04d3100 x1862022114382720/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765859 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775765834.763421:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3100 x1862022114382720/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765859 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765834.763442:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775765834.763455:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775765834.763464:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114382720:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775765834.763475:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114382848:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775765834.763505:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775765834.763519:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765834/real 1775765834]  req@ffff9f14a04d2a00 x1862022114382848/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765859 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775765834.763531:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d2a00 x1862022114382848/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765859 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765834.763540:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775765834.763593:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775765834.763599:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114382848:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775765834.763609:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114382976:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775765834.763639:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775765834.763659:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765834/real 1775765834]  req@ffff9f14a04d0380 x1862022114382976/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765859 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775765834.763671:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0380 x1862022114382976/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765859 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765834.763679:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775765834.763682:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775765834.763684:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114382976:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775765834.763689:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114383104:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775765834.763704:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775765834.763711:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765834/real 1775765834]  req@ffff9f14a04d1f80 x1862022114383104/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765859 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775765834.763721:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1f80 x1862022114383104/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765859 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765834.763728:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775765834.763730:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775765834.763733:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114383104:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775765834.763737:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114383232:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775765834.763750:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775765834.763757:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765834/real 1775765834]  req@ffff9f14a04d3b80 x1862022114383232/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765859 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775765834.763767:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3b80 x1862022114383232/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765859 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765834.763774:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775765834.763777:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775765834.763779:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114383232:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775765834.763799:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114383360:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775765834.763812:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775765834.763819:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765834/real 1775765834]  req@ffff9f14a04d0a80 x1862022114383360/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765859 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775765834.763829:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0a80 x1862022114383360/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765859 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765834.763835:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775765834.763838:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775765834.763840:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114383360:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775765834.763845:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114383488:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775765834.763857:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775765834.763864:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765834/real 1775765834]  req@ffff9f14a04d0e00 x1862022114383488/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765859 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775765834.763874:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0e00 x1862022114383488/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765859 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765834.763887:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775765834.763890:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775765834.763893:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114383488:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775765835.786685:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775765836.042714:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765836.810650:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775765837.066726:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765837.834680:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775765838.090708:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765838.858641:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775765839.114750:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765839.882680:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775765839.882699:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765839.882711:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775765839.882718:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765839.882732:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3217
00000100:00080000:3.0:1775765839.882747:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765839.882756:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775765839.882805:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744a80 x1862022114383616/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765839.882833:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765839.882836:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775765839.882838:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765839.882841:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3217
00000100:00080000:3.0:1775765839.882846:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765839.882850:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765839.882862:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775765839.882872:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114383616:192.168.202.110@tcp:250:kworker.0
00000100:00100000:3.0:1775765839.882877:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745880 x1862022114383744/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765839.882893:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765839.882898:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775765839.882899:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765839.882904:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3217
00000100:00080000:3.0:1775765839.882906:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765839.882908:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765839.882911:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775765839.882921:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746680 x1862022114383872/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765839.882926:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765839.882929:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775765839.882930:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765839.882935:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3217
00000100:00080000:3.0:1775765839.882940:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765839.882942:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765839.882945:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:2.0:1775765839.882950:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775765839.882956:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744e00 x1862022114384000/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765839.882964:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765839.882968:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775765839.882970:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765839.882975:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3217
00000100:00080000:3.0:1775765839.882978:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765839.882982:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765839.882988:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775765839.882995:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765839/real 1775765839]  req@ffff9f1485744a80 x1862022114383616/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765864 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775765839.883000:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747b80 x1862022114384128/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765839.883063:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765839.883066:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775765839.883068:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765839.883078:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3217
00000100:00100000:2.0:1775765839.883079:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744a80 x1862022114383616/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765864 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775765839.883085:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765839.883098:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765839.883112:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00080000:2.0:1775765839.883116:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775765839.883127:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775765839.883127:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746300 x1862022114384256/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775765839.883141:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114383616:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775765839.883151:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765839.883154:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775765839.883156:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765839.883163:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3217
00000100:00080000:3.0:1775765839.883165:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765839.883167:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765839.883170:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775765839.883177:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114383744:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775765839.883182:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746a00 x1862022114384384/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765839.883200:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765839.883206:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775765839.883208:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765839.883212:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3217
00000100:00080000:3.0:1775765839.883219:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000800:00000100:2.0:1775765839.883220:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775765839.883224:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765839.883230:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775765839.883242:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744380 x1862022114384512/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775765839.883246:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765839/real 1775765839]  req@ffff9f1485745880 x1862022114383744/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765864 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775765839.883293:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745880 x1862022114383744/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765864 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765839.883300:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775765839.883306:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775765839.883309:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114383744:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775765839.883315:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114383872:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775765839.883330:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775765839.883336:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765839/real 1775765839]  req@ffff9f1485746680 x1862022114383872/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765864 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775765839.883344:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746680 x1862022114383872/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765864 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765839.883349:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775765839.883351:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775765839.883352:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114383872:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775765839.883356:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114384000:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775765839.883367:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775765839.883373:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765839/real 1775765839]  req@ffff9f1485744e00 x1862022114384000/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765864 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775765839.883380:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744e00 x1862022114384000/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765864 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765839.883389:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775765839.883392:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775765839.883394:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114384000:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775765839.883397:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114384128:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775765839.883406:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775765839.883411:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765839/real 1775765839]  req@ffff9f1485747b80 x1862022114384128/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765864 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775765839.883418:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747b80 x1862022114384128/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765864 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765839.883423:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775765839.883425:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775765839.883427:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114384128:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775765839.883430:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114384256:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775765839.883440:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775765839.883445:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765839/real 1775765839]  req@ffff9f1485746300 x1862022114384256/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765864 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775765839.883451:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746300 x1862022114384256/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765864 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765839.883456:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775765839.883458:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775765839.883462:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114384256:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775765839.883483:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114384384:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775765839.883518:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775765839.883529:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765839/real 1775765839]  req@ffff9f1485746a00 x1862022114384384/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765864 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775765839.883541:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746a00 x1862022114384384/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765864 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765839.883547:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775765839.883549:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775765839.883551:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114384384:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775765839.883574:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114384512:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775765839.883590:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775765839.883597:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765839/real 1775765839]  req@ffff9f1485744380 x1862022114384512/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765864 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775765839.883605:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744380 x1862022114384512/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765864 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765839.883610:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775765839.883612:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775765839.883614:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114384512:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775765840.906602:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775765841.162711:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765841.930648:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:2.0:1775765842.954609:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775765843.210680:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765843.978647:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775765844.234739:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765845.002618:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775765845.002620:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765845.002627:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775765845.002631:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765845.002638:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3222
00000100:00080000:3.0:1775765845.002647:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765845.002653:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775765845.002681:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747480 x1862022114384640/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765845.002702:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765845.002705:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775765845.002706:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765845.002709:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3222
00000100:00080000:3.0:1775765845.002712:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765845.002716:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765845.002724:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775765845.002730:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114384640:192.168.202.110@tcp:250:kworker.0
00000100:00100000:3.0:1775765845.002734:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744700 x1862022114384768/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765845.002743:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765845.002745:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775765845.002747:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765845.002749:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3222
00000100:00080000:3.0:1775765845.002752:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765845.002754:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765845.002757:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775765845.002766:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745500 x1862022114384896/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765845.002772:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765845.002774:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775765845.002775:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765845.002777:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3222
00000100:00080000:3.0:1775765845.002780:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765845.002783:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765845.002786:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775765845.002793:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744000 x1862022114385024/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:2.0:1775765845.002797:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775765845.002809:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765845.002811:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775765845.002813:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765845.002816:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3222
00000100:00080000:3.0:1775765845.002819:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765845.002821:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765845.002824:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775765845.002832:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747800 x1862022114385152/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775765845.002837:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765845/real 1775765845]  req@ffff9f1485747480 x1862022114384640/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765870 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775765845.002843:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765845.002851:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775765845.002855:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765845.002858:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3222
00000100:00080000:3.0:1775765845.002861:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:2.0:1775765845.002864:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747480 x1862022114384640/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765870 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775765845.002867:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765845.002873:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00080000:2.0:1775765845.002882:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775765845.002886:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775765845.002890:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114384640:192.168.202.110@tcp:250:kworker.0
00000100:00100000:3.0:1775765845.002895:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745c00 x1862022114385280/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765845.002903:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765845.002905:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775765845.002906:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765845.002909:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3222
00000100:00100000:2.0:1775765845.002912:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114384768:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775765845.002914:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765845.002921:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765845.002925:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775765845.002935:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745180 x1862022114385408/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:2.0:1775765845.002941:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775765845.002951:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765845.002954:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00100000:2.0:1775765845.002955:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765845/real 1775765845]  req@ffff9f1485744700 x1862022114384768/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765870 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775765845.002958:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765845.002966:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3222
00000100:00080000:3.0:1775765845.002973:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:2.0:1775765845.002978:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744700 x1862022114384768/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765870 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775765845.002978:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765845.002984:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00080000:2.0:1775765845.002992:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775765845.002996:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775765845.002999:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745f80 x1862022114385536/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775765845.003002:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114384768:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775765845.003019:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114384896:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775765845.003034:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775765845.003042:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765845/real 1775765845]  req@ffff9f1485745500 x1862022114384896/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765870 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775765845.003050:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745500 x1862022114384896/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765870 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765845.003055:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775765845.003058:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775765845.003060:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114384896:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775765845.003063:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114385024:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775765845.003073:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775765845.003078:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765845/real 1775765845]  req@ffff9f1485744000 x1862022114385024/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765870 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775765845.003106:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744000 x1862022114385024/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765870 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765845.003113:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775765845.003115:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775765845.003118:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114385024:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775765845.003123:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114385152:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775765845.003138:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775765845.003161:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765845/real 1775765845]  req@ffff9f1485747800 x1862022114385152/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765870 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775765845.003170:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747800 x1862022114385152/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765870 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765845.003175:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775765845.003176:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775765845.003178:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114385152:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775765845.003181:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114385280:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775765845.003192:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775765845.003197:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765845/real 1775765845]  req@ffff9f1485745c00 x1862022114385280/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765870 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775765845.003203:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745c00 x1862022114385280/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765870 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765845.003208:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775765845.003210:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775765845.003212:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114385280:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775765845.003232:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114385408:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775765845.003244:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775765845.003250:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765845/real 1775765845]  req@ffff9f1485745180 x1862022114385408/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765870 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775765845.003256:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745180 x1862022114385408/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765870 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765845.003266:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775765845.003268:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775765845.003270:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114385408:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775765845.003274:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114385536:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775765845.003283:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775765845.003288:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765845/real 1775765845]  req@ffff9f1485745f80 x1862022114385536/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765870 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775765845.003295:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745f80 x1862022114385536/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765870 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765845.003300:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775765845.003302:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775765845.003303:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114385536:192.168.202.110@tcp:8:kworker.0
00010000:00010000:3.0:1775765845.258841:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765846.026638:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775765846.282736:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765847.050650:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:2.0:1775765848.074630:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775765848.330779:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765849.098630:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:2.0:1775765850.122654:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:1.0:1775765850.122659:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765850.122666:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1775765850.122671:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765850.122679:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3227
00000100:00080000:1.0:1775765850.122689:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765850.122696:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775765850.122735:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdc00 x1862022114385664/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765850.122767:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765850.122771:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775765850.122772:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765850.122777:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3227
00000100:00080000:1.0:1775765850.122781:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765850.122785:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765850.122794:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775765850.122795:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114385664:192.168.202.110@tcp:250:kworker.0
00000100:00100000:1.0:1775765850.122809:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdf80 x1862022114385792/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765850.122818:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765850.122821:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775765850.122822:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765850.122826:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3227
00000100:00080000:1.0:1775765850.122828:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765850.122832:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765850.122835:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775765850.122846:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fd880 x1862022114385920/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765850.122856:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765850.122860:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775765850.122862:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000800:00000100:2.0:1775765850.122868:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:1.0:1775765850.122872:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3227
00000100:00080000:1.0:1775765850.122878:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765850.122884:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765850.122890:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775765850.122905:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fce00 x1862022114386048/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775765850.122910:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765850/real 1775765850]  req@ffff9f14ab4fdc00 x1862022114385664/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765875 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765850.122932:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765850.122941:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1775765850.122944:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765850.122952:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3227
00000100:00080000:1.0:1775765850.122956:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00100000:2.0:1775765850.122959:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdc00 x1862022114385664/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765875 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765850.122964:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765850.122973:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00080000:2.0:1775765850.122975:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775765850.122981:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775765850.122985:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114385664:192.168.202.110@tcp:250:kworker.0
00000100:00100000:1.0:1775765850.122988:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ffb80 x1862022114386176/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765850.122997:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765850.123000:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775765850.123001:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765850.123004:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3227
00000100:00080000:1.0:1775765850.123007:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:2.0:1775765850.123010:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114385792:192.168.202.110@tcp:38:kworker.0
00000100:00080000:1.0:1775765850.123012:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765850.123019:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775765850.123032:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff800 x1862022114386304/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:2.0:1775765850.123040:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:1.0:1775765850.123042:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765850.123046:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775765850.123047:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765850.123052:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3227
00000100:00100000:2.0:1775765850.123055:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765850/real 1775765850]  req@ffff9f14ab4fdf80 x1862022114385792/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765875 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765850.123057:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765850.123065:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765850.123072:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775765850.123077:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdf80 x1862022114385792/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765875 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765850.123084:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fed80 x1862022114386432/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775765850.123086:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775765850.123095:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00080000:1.0:1775765850.123101:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:2.0:1775765850.123102:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114385792:192.168.202.110@tcp:38:kworker.0
00000100:00080000:1.0:1775765850.123107:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775765850.123110:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:2.0:1775765850.123112:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114385920:192.168.202.110@tcp:38:kworker.0
00000100:00080000:1.0:1775765850.123116:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3227
00000100:00080000:1.0:1775765850.123120:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765850.123123:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765850.123128:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:2.0:1775765850.123134:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775765850.123141:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fca80 x1862022114386560/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775765850.123145:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765850/real 1775765850]  req@ffff9f14ab4fd880 x1862022114385920/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765875 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775765850.123168:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fd880 x1862022114385920/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765875 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765850.123183:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775765850.123188:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775765850.123191:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114385920:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775765850.123198:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114386048:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775765850.123221:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775765850.123233:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765850/real 1775765850]  req@ffff9f14ab4fce00 x1862022114386048/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765875 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775765850.123247:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fce00 x1862022114386048/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765875 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765850.123255:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775765850.123257:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775765850.123259:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114386048:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775765850.123272:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ffb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114386176:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775765850.123306:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775765850.123315:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765850/real 1775765850]  req@ffff9f14ab4ffb80 x1862022114386176/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765875 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775765850.123324:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ffb80 x1862022114386176/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765875 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765850.123330:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775765850.123333:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775765850.123336:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ffb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114386176:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775765850.123351:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114386304:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775765850.123372:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775765850.123380:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765850/real 1775765850]  req@ffff9f14ab4ff800 x1862022114386304/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765875 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775765850.123389:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff800 x1862022114386304/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765875 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765850.123437:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775765850.123440:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775765850.123443:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114386304:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775765850.123447:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114386432:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775765850.123463:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775765850.123471:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765850/real 1775765850]  req@ffff9f14ab4fed80 x1862022114386432/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765875 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775765850.123479:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fed80 x1862022114386432/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765875 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765850.123485:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775765850.123487:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775765850.123489:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114386432:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775765850.123493:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114386560:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775765850.123502:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775765850.123508:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765850/real 1775765850]  req@ffff9f14ab4fca80 x1862022114386560/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765875 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775765850.123516:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fca80 x1862022114386560/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765875 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765850.123521:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775765850.123523:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775765850.123525:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114386560:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775765850.378746:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765851.146672:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775765851.402790:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765852.170690:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775765852.426744:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765853.194654:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775765853.450652:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765854.218654:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:2.0:1775765855.242650:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775765855.242670:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765855.242677:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775765855.242682:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765855.242693:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3232
00000100:00080000:3.0:1775765855.242704:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765855.242709:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775765855.242743:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747100 x1862022114386688/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765855.242767:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765855.242770:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775765855.242771:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765855.242775:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3232
00000100:00080000:3.0:1775765855.242777:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765855.242780:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765855.242787:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775765855.242798:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745f80 x1862022114386816/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775765855.242806:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114386688:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775765855.242809:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765855.242815:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775765855.242818:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765855.242822:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3232
00000100:00080000:3.0:1775765855.242824:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765855.242826:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765855.242829:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775765855.242838:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745180 x1862022114386944/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765855.242845:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765855.242847:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775765855.242848:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765855.242851:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3232
00000100:00080000:3.0:1775765855.242861:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765855.242864:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765855.242868:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:2.0:1775765855.242875:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775765855.242879:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745c00 x1862022114387072/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765855.242886:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765855.242888:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775765855.242890:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765855.242893:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3232
00000100:00080000:3.0:1775765855.242895:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765855.242898:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765855.242901:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775765855.242908:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747800 x1862022114387200/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765855.242914:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765855.242918:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00100000:2.0:1775765855.242919:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765855/real 1775765855]  req@ffff9f1485747100 x1862022114386688/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765880 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775765855.242920:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765855.242927:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3232
00000100:00080000:3.0:1775765855.242931:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765855.242937:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765855.242942:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775765855.242951:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747100 x1862022114386688/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765880 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775765855.242952:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744000 x1862022114387328/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775765855.242981:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775765855.242990:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775765855.242998:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114386688:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775765855.243000:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765855.243006:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775765855.243009:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765855.243015:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3232
00000100:00080000:3.0:1775765855.243019:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:2.0:1775765855.243022:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114386816:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775765855.243028:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765855.243035:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000800:00000100:2.0:1775765855.243047:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775765855.243049:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745500 x1862022114387456/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765855.243060:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:2.0:1775765855.243062:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765855/real 1775765855]  req@ffff9f1485745f80 x1862022114386816/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765880 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775765855.243069:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775765855.243074:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765855.243082:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3232
00000100:00080000:3.0:1775765855.243088:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:2.0:1775765855.243091:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745f80 x1862022114386816/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765880 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775765855.243094:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765855.243104:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00080000:2.0:1775765855.243105:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775765855.243112:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775765855.243115:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114386816:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775765855.243124:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114386944:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775765855.243125:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744700 x1862022114387584/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:2.0:1775765855.243146:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775765855.243154:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765855/real 1775765855]  req@ffff9f1485745180 x1862022114386944/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765880 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775765855.243164:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745180 x1862022114386944/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765880 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765855.243170:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775765855.243172:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775765855.243174:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114386944:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775765855.243178:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114387072:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775765855.243188:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775765855.243194:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765855/real 1775765855]  req@ffff9f1485745c00 x1862022114387072/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765880 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775765855.243206:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745c00 x1862022114387072/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765880 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765855.243211:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775765855.243213:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775765855.243215:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114387072:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775765855.243224:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114387200:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775765855.243236:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775765855.243242:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765855/real 1775765855]  req@ffff9f1485747800 x1862022114387200/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765880 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775765855.243252:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747800 x1862022114387200/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765880 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765855.243260:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775765855.243264:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775765855.243267:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114387200:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775765855.243274:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114387328:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775765855.243295:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775765855.243306:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765855/real 1775765855]  req@ffff9f1485744000 x1862022114387328/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765880 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775765855.243318:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744000 x1862022114387328/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765880 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765855.243328:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775765855.243331:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775765855.243333:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114387328:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775765855.243383:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114387456:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775765855.243407:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775765855.243418:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765855/real 1775765855]  req@ffff9f1485745500 x1862022114387456/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765880 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775765855.243427:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745500 x1862022114387456/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765880 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765855.243433:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775765855.243435:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775765855.243437:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114387456:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775765855.243441:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114387584:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775765855.243454:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775765855.243460:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765855/real 1775765855]  req@ffff9f1485744700 x1862022114387584/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765880 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775765855.243467:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744700 x1862022114387584/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765880 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765855.243472:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775765855.243474:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775765855.243476:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114387584:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775765855.498705:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765856.266630:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:2.0:1775765857.290621:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775765857.546677:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765858.314652:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775765858.570748:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765859.338646:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775765859.594727:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765860.362622:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:1.0:1775765860.362686:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765860.362691:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1775765860.362695:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765860.362703:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3237
00000100:00080000:1.0:1775765860.362714:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765860.362721:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775765860.362788:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fc380 x1862022114387712/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765860.362819:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765860.362822:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775765860.362824:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765860.362828:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3237
00000100:00080000:1.0:1775765860.362833:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765860.362837:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765860.362848:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775765860.362851:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114387712:192.168.202.110@tcp:250:kworker.0
00000100:00100000:1.0:1775765860.362864:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fea00 x1862022114387840/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765860.362874:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765860.362877:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775765860.362879:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765860.362882:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3237
00000100:00080000:1.0:1775765860.362886:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765860.362890:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765860.362894:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775765860.362903:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff100 x1862022114387968/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765860.362911:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000800:00000100:2.0:1775765860.362916:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:1.0:1775765860.362917:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775765860.362920:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765860.362925:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3237
00000100:00080000:1.0:1775765860.362929:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765860.362932:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765860.362937:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775765860.362954:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fe300 x1862022114388096/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765860.362964:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:2.0:1775765860.362964:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765860/real 1775765860]  req@ffff9f14ab4fc380 x1862022114387712/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765885 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765860.362973:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1775765860.362976:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765860.362984:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3237
00000100:00080000:1.0:1775765860.362988:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765860.362994:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765860.362998:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775765860.363007:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fc380 x1862022114387712/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765885 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765860.363009:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fca80 x1862022114388224/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775765860.363057:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765860.363064:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765860.363066:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:1.0:1775765860.363068:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775765860.363070:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:2.0:1775765860.363072:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114387712:192.168.202.110@tcp:250:kworker.0
00000100:00080000:1.0:1775765860.363076:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3237
00000100:00080000:1.0:1775765860.363103:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765860.363106:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765860.363110:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775765860.363116:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114387840:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765860.363122:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fed80 x1862022114388352/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765860.363131:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765860.363134:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775765860.363136:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765860.363139:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3237
00000100:00080000:1.0:1775765860.363142:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000800:00000100:2.0:1775765860.363147:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:1.0:1775765860.363150:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765860.363156:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775765860.363167:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765860/real 1775765860]  req@ffff9f14ab4fea00 x1862022114387840/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765885 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765860.363168:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff800 x1862022114388480/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765860.363216:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765860.363221:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00100000:2.0:1775765860.363221:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fea00 x1862022114387840/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765885 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765860.363225:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765860.363235:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3237
00000100:00080000:1.0:1775765860.363243:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765860.363249:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00080000:2.0:1775765860.363250:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000008:00100000:1.0:1775765860.363253:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00080000:2.0:1775765860.363276:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775765860.363283:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114387840:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765860.363291:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ffb80 x1862022114388608/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775765860.363293:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114387968:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775765860.363360:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775765860.363369:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765860/real 1775765860]  req@ffff9f14ab4ff100 x1862022114387968/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765885 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775765860.363378:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff100 x1862022114387968/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765885 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765860.363384:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775765860.363386:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775765860.363388:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114387968:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775765860.363392:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114388096:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775765860.363404:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775765860.363409:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765860/real 1775765860]  req@ffff9f14ab4fe300 x1862022114388096/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765885 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775765860.363442:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fe300 x1862022114388096/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765885 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765860.363449:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775765860.363452:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775765860.363454:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114388096:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775765860.363459:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114388224:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775765860.363473:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775765860.363480:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765860/real 1775765860]  req@ffff9f14ab4fca80 x1862022114388224/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765885 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775765860.363489:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fca80 x1862022114388224/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765885 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765860.363495:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775765860.363497:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775765860.363499:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114388224:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775765860.363514:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114388352:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775765860.363525:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775765860.363532:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765860/real 1775765860]  req@ffff9f14ab4fed80 x1862022114388352/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765885 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775765860.363541:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fed80 x1862022114388352/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765885 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765860.363547:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775765860.363549:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775765860.363552:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114388352:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775765860.363575:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114388480:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775765860.363589:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775765860.363597:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765860/real 1775765860]  req@ffff9f14ab4ff800 x1862022114388480/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765885 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775765860.363611:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff800 x1862022114388480/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765885 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765860.363617:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775765860.363620:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775765860.363622:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114388480:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775765860.363626:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ffb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114388608:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775765860.363636:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775765860.363642:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765860/real 1775765860]  req@ffff9f14ab4ffb80 x1862022114388608/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765885 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775765860.363667:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ffb80 x1862022114388608/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765885 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765860.363672:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775765860.363673:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775765860.363675:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ffb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114388608:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775765860.618682:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765861.386713:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:2.0:1775765862.410620:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775765862.666724:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00010000:00010000:2.0:1775765864.714700:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1775765865.482663:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765865.482669:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775765865.482674:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765865.482682:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3242
00000100:00080000:3.0:1775765865.482691:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765865.482698:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775765865.482738:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747480 x1862022114388736/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765865.482765:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765865.482768:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775765865.482770:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765865.482773:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3242
00000100:00080000:3.0:1775765865.482776:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765865.482779:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765865.482788:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775765865.482799:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744380 x1862022114388864/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775765865.482806:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114388736:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775765865.482809:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765865.482815:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775765865.482816:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765865.482819:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3242
00000100:00080000:3.0:1775765865.482822:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765865.482824:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765865.482828:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775765865.482835:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746a00 x1862022114388992/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765865.482846:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765865.482849:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775765865.482850:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765865.482852:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3242
00000100:00080000:3.0:1775765865.482855:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765865.482857:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765865.482860:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775765865.482867:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746300 x1862022114389120/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765865.482877:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765865.482880:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775765865.482882:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765865.482887:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3242
00000100:00080000:3.0:1775765865.482891:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765865.482896:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775765865.482898:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114388864:192.168.202.110@tcp:38:kworker.0
00000008:00100000:3.0:1775765865.482904:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775765865.482920:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747b80 x1862022114389248/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765865.482931:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765865.482934:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775765865.482937:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:2.0:1775765865.482938:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114388992:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775765865.482945:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3242
00000100:00080000:3.0:1775765865.482951:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765865.482955:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765865.482961:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775765865.482974:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114389120:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775765865.482985:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744e00 x1862022114389376/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765865.482995:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765865.482999:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775765865.483001:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:2.0:1775765865.483006:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114389248:192.168.202.110@tcp:8:kworker.0
00000100:00080000:3.0:1775765865.483007:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3242
00000100:00080000:3.0:1775765865.483015:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765865.483020:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765865.483026:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775765865.483030:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114389376:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775765865.483041:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746680 x1862022114389504/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765865.483052:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765865.483056:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775765865.483058:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:2.0:1775765865.483062:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114389504:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775765865.483064:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3242
00000100:00080000:3.0:1775765865.483072:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765865.483077:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765865.483083:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775765865.483094:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745880 x1862022114389632/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775765865.483140:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114389632:192.168.202.110@tcp:8:kworker.0
00010000:00010000:3.0:1775765865.738778:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00010000:00010000:3.0:1775765866.762737:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00010000:00010000:2.0:1775765867.786737:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000400:00000100:2.0:1775765868.554870:0:2391:0:(lib-socket.c:454:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.202.110:988
00000400:00000100:2.0:1775765868.554911:0:2391:0:(acceptor.c:111:lnet_connect_console_error()) Connection to 192.168.202.110@tcp at host 192.168.202.110 was unreachable: the network or that node may be down, or Lustre may be misconfigured.
00000800:00000100:2.0:1775765868.554928:0:2391:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:2.0:1775765868.554969:0:2391:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:2.0:1775765868.554978:0:2391:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:2.0:1775765868.554986:0:2391:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:2.0:1775765868.554993:0:2391:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:2.0:1775765868.555001:0:2391:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:2.0:1775765868.555008:0:2391:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:2.0:1775765868.555015:0:2391:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00010000:00010000:2.0:1775765869.834709:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1775765870.602639:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765870.602648:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775765870.602652:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765870.602662:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3248
00000100:00080000:3.0:1775765870.602672:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765870.602681:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775765870.602720:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744a80 x1862022114389760/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765870.602750:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765870.602755:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775765870.602757:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765870.602763:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3248
00000100:00080000:3.0:1775765870.602769:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765870.602774:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765870.602784:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775765870.602797:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747480 x1862022114389888/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765870.602806:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765870.602809:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775765870.602810:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765870.602814:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3248
00000100:00080000:3.0:1775765870.602818:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765870.602822:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765870.602826:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775765870.602836:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746300 x1862022114390016/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765870.602849:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765870.602852:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775765870.602853:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765870.602856:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3248
00000100:00080000:3.0:1775765870.602858:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765870.602862:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765870.602866:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775765870.602883:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746a00 x1862022114390144/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765870.602890:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765870.602893:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775765870.602894:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765870.602895:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3248
00000100:00080000:3.0:1775765870.602898:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765870.602900:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765870.602904:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775765870.602912:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744380 x1862022114390272/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765870.602931:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765870.602933:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775765870.602935:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765870.602939:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3248
00000100:00080000:3.0:1775765870.602942:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765870.602946:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765870.602950:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775765870.602959:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744e00 x1862022114390400/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765870.602966:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765870.602968:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775765870.602969:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765870.602973:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3248
00000100:00080000:3.0:1775765870.602976:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765870.602981:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765870.602984:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775765870.602996:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747b80 x1862022114390528/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765870.603003:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765870.603006:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775765870.603009:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765870.603014:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3248
00000100:00080000:3.0:1775765870.603019:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765870.603025:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765870.603032:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775765870.603040:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746680 x1862022114390656/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00010000:00010000:2.0:1775765871.882789:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00010000:00010000:3.0:1775765872.906699:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00010000:00010000:3.0:1775765873.930743:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00010000:00010000:2.0:1775765874.954718:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1775765875.722685:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765875.722695:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775765875.722699:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765875.722706:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3253
00000100:00080000:3.0:1775765875.722715:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765875.722721:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775765875.722755:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745880 x1862022114390784/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765875.722783:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765875.722785:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775765875.722787:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765875.722789:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3253
00000100:00080000:3.0:1775765875.722792:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765875.722794:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765875.722801:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775765875.722810:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744700 x1862022114390912/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765875.722824:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765875.722828:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775765875.722830:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765875.722833:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3253
00000100:00080000:3.0:1775765875.722835:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765875.722837:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765875.722840:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775765875.722848:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745500 x1862022114391040/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765875.722854:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765875.722856:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775765875.722857:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765875.722860:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3253
00000100:00080000:3.0:1775765875.722862:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765875.722865:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765875.722868:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775765875.722876:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744000 x1862022114391168/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765875.722887:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765875.722889:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775765875.722891:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765875.722895:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3253
00000100:00080000:3.0:1775765875.722898:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765875.722901:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765875.722904:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775765875.722911:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747800 x1862022114391296/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765875.722918:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765875.722922:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775765875.722925:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765875.722933:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3253
00000100:00080000:3.0:1775765875.722938:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765875.722941:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765875.722945:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775765875.722957:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745c00 x1862022114391424/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765875.722970:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765875.722973:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775765875.722976:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765875.722984:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3253
00000100:00080000:3.0:1775765875.722987:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765875.722989:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765875.722992:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775765875.722998:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745180 x1862022114391552/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765875.723012:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765875.723017:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775765875.723019:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765875.723023:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3253
00000100:00080000:3.0:1775765875.723025:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765875.723027:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765875.723031:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775765875.723039:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745f80 x1862022114391680/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00010000:00010000:2.0:1775765877.002757:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00010000:00010000:2.0:1775765879.050698:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00010000:00010000:3.0:1775765880.074731:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1775765880.842708:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765880.842730:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1775765880.842735:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765880.842749:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3258
00000100:00080000:2.0:1775765880.842765:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765880.842775:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775765880.842827:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1180 x1862022114391808/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775765880.842863:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765880.842867:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775765880.842869:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765880.842877:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3258
00000100:00080000:2.0:1775765880.842882:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765880.842888:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775765880.842902:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775765880.842930:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1f80 x1862022114391936/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775765880.842946:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765880.842951:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775765880.842953:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765880.842958:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3258
00000100:00080000:2.0:1775765880.842965:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765880.842970:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775765880.842985:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775765880.843007:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2a00 x1862022114392064/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775765880.843021:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765880.843028:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775765880.843034:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765880.843052:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3258
00000100:00080000:2.0:1775765880.843072:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765880.843087:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775765880.843095:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775765880.843126:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2680 x1862022114392192/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775765880.843142:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765880.843151:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1775765880.843154:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765880.843161:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3258
00000100:00080000:2.0:1775765880.843169:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765880.843175:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775765880.843182:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775765880.843207:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1500 x1862022114392320/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775765880.843262:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765880.843271:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775765880.843274:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765880.843284:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3258
00000100:00080000:2.0:1775765880.843293:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765880.843304:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775765880.843312:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775765880.843336:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0a80 x1862022114392448/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775765880.843356:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765880.843360:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775765880.843365:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765880.843383:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3258
00000100:00080000:2.0:1775765880.843390:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765880.843403:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775765880.843412:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775765880.843436:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0700 x1862022114392576/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775765880.843461:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765880.843466:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775765880.843468:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765880.843475:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3258
00000100:00080000:2.0:1775765880.843481:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765880.843495:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775765880.843503:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775765880.843526:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2d80 x1862022114392704/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00010000:00010000:3.0:1775765881.098726:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00010000:00010000:2.0:1775765882.122714:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00010000:00010000:2.0:1775765884.170747:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1775765885.962680:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765885.962688:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1775765885.962693:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765885.962705:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3263
00000100:00080000:1.0:1775765885.962719:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765885.962728:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775765885.962781:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fce00 x1862022114392832/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765885.962807:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765885.962811:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775765885.962812:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765885.962816:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3263
00000100:00080000:1.0:1775765885.962820:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765885.962824:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765885.962835:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775765885.962854:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fd880 x1862022114392960/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765885.962868:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765885.962876:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775765885.962878:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765885.962882:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3263
00000100:00080000:1.0:1775765885.962887:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765885.962891:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765885.962897:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775765885.962907:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdf80 x1862022114393088/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765885.962915:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765885.962918:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775765885.962920:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765885.962923:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3263
00000100:00080000:1.0:1775765885.962926:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765885.962929:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765885.962935:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775765885.962946:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdc00 x1862022114393216/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765885.962953:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765885.962957:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1775765885.962959:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765885.962962:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3263
00000100:00080000:1.0:1775765885.962964:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765885.962967:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765885.962971:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775765885.962979:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ffb80 x1862022114393344/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765885.962987:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765885.962990:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775765885.962992:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765885.962997:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3263
00000100:00080000:1.0:1775765885.963002:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765885.963007:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765885.963014:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775765885.963024:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff800 x1862022114393472/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765885.963031:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765885.963035:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775765885.963036:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765885.963076:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3263
00000100:00080000:1.0:1775765885.963081:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765885.963087:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765885.963093:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775765885.963106:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fed80 x1862022114393600/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765885.963114:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765885.963117:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775765885.963119:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765885.963122:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3263
00000100:00080000:1.0:1775765885.963125:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765885.963130:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765885.963134:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775765885.963146:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fca80 x1862022114393728/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00010000:00010000:2.0:1775765886.218775:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00010000:00010000:3.0:1775765887.242788:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00010000:00010000:3.0:1775765888.266749:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00010000:00010000:2.0:1775765889.290696:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765891.082709:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:2.0:1775765891.082801:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114393856:192.168.202.110@tcp:250:kworker.0
00000800:00000100:2.0:1775765891.082855:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775765891.082902:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765891/real 1775765891]  req@ffff9f14a04d3800 x1862022114393856/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765916 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775765891.082922:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3800 x1862022114393856/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765916 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765891.082939:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775765891.082945:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775765891.082953:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114393856:192.168.202.110@tcp:250:kworker.0
00000100:00100000:2.0:1775765891.082979:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114393984:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775765891.083001:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775765891.083033:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765891/real 1775765891]  req@ffff9f14a04d2300 x1862022114393984/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765916 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775765891.083051:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d2300 x1862022114393984/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765916 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765891.083062:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775765891.083068:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775765891.083072:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114393984:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775765891.083083:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114394112:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775765891.083106:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775765891.083121:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765891/real 1775765891]  req@ffff9f14a04d1180 x1862022114394112/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765916 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775765891.083138:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1180 x1862022114394112/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765916 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765891.083147:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775765891.083150:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775765891.083154:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114394112:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775765891.083161:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114394240:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775765891.083182:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775765891.083192:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765891/real 1775765891]  req@ffff9f14a04d3480 x1862022114394240/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765916 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775765891.083204:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3480 x1862022114394240/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765916 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765891.083211:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775765891.083214:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775765891.083217:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114394240:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775765891.083223:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114394368:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775765891.083240:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775765891.083251:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765891/real 1775765891]  req@ffff9f14a04d0000 x1862022114394368/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765916 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775765891.083261:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0000 x1862022114394368/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765916 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765891.083266:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775765891.083268:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775765891.083270:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114394368:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775765891.083283:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114394496:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775765891.083297:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775765891.083303:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765891/real 1775765891]  req@ffff9f14a04d1500 x1862022114394496/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765916 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775765891.083311:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1500 x1862022114394496/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765916 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765891.083316:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775765891.083318:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775765891.083320:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114394496:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775765891.083323:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114394624:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775765891.083332:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775765891.083337:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765891/real 1775765891]  req@ffff9f14a04d3800 x1862022114394624/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765916 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775765891.083344:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3800 x1862022114394624/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765916 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765891.083350:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775765891.083352:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775765891.083354:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114394624:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775765891.083357:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114394752:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775765891.083366:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775765891.083371:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765891/real 1775765891]  req@ffff9f14a04d0e00 x1862022114394752/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765916 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775765891.083377:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0e00 x1862022114394752/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765916 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765891.083382:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775765891.083384:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775765891.083386:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114394752:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775765891.338718:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00010000:00010000:2.0:1775765893.386713:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00010000:00010000:3.0:1775765894.410724:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00010000:00010000:3.0:1775765895.434718:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765896.202706:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:2.0:1775765896.202831:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114394880:192.168.202.110@tcp:250:kworker.0
00000800:00000100:2.0:1775765896.202898:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775765896.202973:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765896/real 1775765896]  req@ffff9f14a04d0a80 x1862022114394880/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765921 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775765896.202993:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0a80 x1862022114394880/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765921 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765896.203038:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775765896.203046:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775765896.203053:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114394880:192.168.202.110@tcp:250:kworker.0
00000100:00100000:2.0:1775765896.203081:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114395008:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775765896.203107:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775765896.203124:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765896/real 1775765896]  req@ffff9f14a04d3b80 x1862022114395008/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765921 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775765896.203156:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3b80 x1862022114395008/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765921 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765896.203172:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775765896.203177:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775765896.203182:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114395008:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775765896.203194:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114395136:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775765896.203221:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775765896.203232:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765896/real 1775765896]  req@ffff9f14a04d1f80 x1862022114395136/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765921 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775765896.203243:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1f80 x1862022114395136/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765921 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765896.203249:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775765896.203252:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775765896.203254:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114395136:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775765896.203262:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114395264:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775765896.203285:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775765896.203298:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765896/real 1775765896]  req@ffff9f14a04d0380 x1862022114395264/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765921 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775765896.203313:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0380 x1862022114395264/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765921 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765896.203322:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775765896.203325:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775765896.203328:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114395264:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775765896.203333:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114395392:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775765896.203348:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775765896.203355:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765896/real 1775765896]  req@ffff9f14a04d2a00 x1862022114395392/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765921 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775765896.203362:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d2a00 x1862022114395392/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765921 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765896.203367:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775765896.203369:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775765896.203371:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114395392:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775765896.203384:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114395520:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775765896.203394:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775765896.203411:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765896/real 1775765896]  req@ffff9f14a04d3100 x1862022114395520/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765921 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775765896.203424:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3100 x1862022114395520/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765921 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765896.203433:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775765896.203436:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775765896.203440:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114395520:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775765896.203446:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114395648:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775765896.203466:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775765896.203474:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765896/real 1775765896]  req@ffff9f14a04d0a80 x1862022114395648/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765921 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775765896.203482:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0a80 x1862022114395648/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765921 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765896.203487:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775765896.203489:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775765896.203491:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114395648:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775765896.203495:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114395776:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775765896.203505:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775765896.203510:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765896/real 1775765896]  req@ffff9f14a04d0e00 x1862022114395776/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765921 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775765896.203517:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0e00 x1862022114395776/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765921 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765896.203527:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775765896.203529:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775765896.203531:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114395776:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775765896.458686:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765897.226613:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:2.0:1775765898.250693:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775765898.506777:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765899.274760:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:2.0:1775765900.298628:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775765900.554717:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765901.322653:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:2.0:1775765901.322813:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114395904:192.168.202.110@tcp:250:kworker.0
00000800:00000100:2.0:1775765901.322978:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775765901.323071:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765901/real 1775765901]  req@ffff9f14a04d3800 x1862022114395904/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765926 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775765901.323092:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3800 x1862022114395904/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765926 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765901.323103:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775765901.323107:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775765901.323111:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114395904:192.168.202.110@tcp:250:kworker.0
00000100:00100000:2.0:1775765901.323149:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114396032:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775765901.323172:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775765901.323182:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765901/real 1775765901]  req@ffff9f14a04d1500 x1862022114396032/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765926 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775765901.323198:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1500 x1862022114396032/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765926 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765901.323205:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775765901.323207:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775765901.323210:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114396032:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775765901.323225:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114396160:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775765901.323239:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775765901.323246:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765901/real 1775765901]  req@ffff9f14a04d0000 x1862022114396160/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765926 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775765901.323255:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0000 x1862022114396160/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765926 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765901.323261:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775765901.323263:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775765901.323265:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114396160:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775765901.323269:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114396288:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775765901.323282:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775765901.323294:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765901/real 1775765901]  req@ffff9f14a04d3480 x1862022114396288/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765926 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775765901.323303:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3480 x1862022114396288/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765926 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765901.323314:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775765901.323316:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775765901.323319:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114396288:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775765901.323323:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114396416:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775765901.323335:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775765901.323342:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765901/real 1775765901]  req@ffff9f14a04d1180 x1862022114396416/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765926 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775765901.323349:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1180 x1862022114396416/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765926 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765901.323355:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775765901.323358:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775765901.323360:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114396416:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775765901.323365:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114396544:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775765901.323376:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775765901.323383:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765901/real 1775765901]  req@ffff9f14a04d2300 x1862022114396544/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765926 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775765901.323391:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d2300 x1862022114396544/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765926 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765901.323397:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775765901.323399:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775765901.323402:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114396544:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775765901.323407:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114396672:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775765901.323419:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775765901.323425:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765901/real 1775765901]  req@ffff9f14a04d0e00 x1862022114396672/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765926 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775765901.323434:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0e00 x1862022114396672/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765926 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765901.323440:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775765901.323442:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775765901.323444:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114396672:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775765901.323448:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114396800:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775765901.323459:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775765901.323465:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765901/real 1775765901]  req@ffff9f14a04d0a80 x1862022114396800/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765926 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775765901.323473:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0a80 x1862022114396800/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765926 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765901.323492:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775765901.323494:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775765901.323496:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114396800:192.168.202.110@tcp:8:kworker.0
00010000:00010000:3.0:1775765901.578744:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765902.346668:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775765902.602740:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765903.370646:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775765903.626729:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765904.394662:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:2.0:1775765905.418692:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775765905.674748:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765906.442654:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:2.0:1775765906.442790:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114396928:192.168.202.110@tcp:250:kworker.0
00000800:00000100:2.0:1775765906.442900:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775765906.442940:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765906/real 1775765906]  req@ffff9f14a04d3100 x1862022114396928/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765931 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775765906.442965:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3100 x1862022114396928/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765931 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765906.442992:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775765906.442998:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775765906.443005:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114396928:192.168.202.110@tcp:250:kworker.0
00000100:00100000:2.0:1775765906.443029:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114397056:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775765906.443073:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775765906.443100:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765906/real 1775765906]  req@ffff9f14a04d2a00 x1862022114397056/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765931 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775765906.443137:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d2a00 x1862022114397056/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765931 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765906.443171:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775765906.443178:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775765906.443185:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114397056:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775765906.443196:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114397184:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775765906.443236:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775765906.443245:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765906/real 1775765906]  req@ffff9f14a04d0380 x1862022114397184/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765931 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775765906.443256:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0380 x1862022114397184/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765931 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765906.443264:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775765906.443267:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775765906.443271:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114397184:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775765906.443276:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114397312:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775765906.443296:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775765906.443313:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765906/real 1775765906]  req@ffff9f14a04d1f80 x1862022114397312/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765931 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775765906.443328:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1f80 x1862022114397312/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765931 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765906.443340:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775765906.443344:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775765906.443348:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114397312:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775765906.443358:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114397440:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775765906.443389:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775765906.443407:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765906/real 1775765906]  req@ffff9f14a04d3b80 x1862022114397440/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765931 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775765906.443425:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3b80 x1862022114397440/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765931 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765906.443434:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775765906.443438:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775765906.443442:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114397440:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775765906.443464:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114397568:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775765906.443489:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775765906.443502:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765906/real 1775765906]  req@ffff9f14a04d0a80 x1862022114397568/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765931 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775765906.443517:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0a80 x1862022114397568/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765931 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765906.443527:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775765906.443531:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775765906.443535:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114397568:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775765906.443539:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114397696:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775765906.443559:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775765906.443569:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765906/real 1775765906]  req@ffff9f14a04d0e00 x1862022114397696/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765931 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775765906.443580:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0e00 x1862022114397696/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765931 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765906.443628:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775765906.443633:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775765906.443639:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114397696:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775765906.443647:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114397824:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775765906.443693:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775765906.443704:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765906/real 1775765906]  req@ffff9f14a04d2300 x1862022114397824/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765931 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775765906.443714:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d2300 x1862022114397824/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765931 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765906.443720:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775765906.443722:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775765906.443724:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114397824:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775765907.466646:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775765907.722721:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765908.490675:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775765908.746870:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765909.514612:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775765909.770744:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765910.538647:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775765910.794772:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1775765911.562795:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765911.562834:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1775765911.562842:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765911.562859:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3289
00000100:00080000:2.0:1775765911.562878:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765911.562892:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775765911.562957:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3800 x1862022114397952/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775765911.563008:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765911.563013:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775765911.563015:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765911.563021:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3289
00000100:00080000:2.0:1775765911.563026:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765911.563033:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775765911.563048:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775765911.563070:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0380 x1862022114398080/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775765911.563091:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765911.563095:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775765911.563097:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765911.563103:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3289
00000100:00080000:2.0:1775765911.563108:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765911.563112:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775765911.563118:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775765911.563133:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0000 x1862022114398208/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775765911.563144:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765911.563150:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775765911.563153:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765911.563158:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3289
00000100:00080000:2.0:1775765911.563162:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765911.563167:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775765911.563174:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775765911.563188:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3b80 x1862022114398336/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775765911.563199:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765911.563205:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1775765911.563212:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765911.563221:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3289
00000100:00080000:2.0:1775765911.563227:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765911.563233:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775765911.563238:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775765911.563251:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0e00 x1862022114398464/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775765911.563262:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765911.563266:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775765911.563269:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765911.563274:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3289
00000100:00080000:2.0:1775765911.563279:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765911.563285:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775765911.563297:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775765911.563317:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1880 x1862022114398592/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775765911.563366:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765911.563371:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775765911.563373:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765911.563377:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3289
00000100:00080000:2.0:1775765911.563381:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765911.563384:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775765911.563389:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775765911.563406:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3100 x1862022114398720/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775765911.563464:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765911.563469:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775765911.563473:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765911.563486:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3289
00000100:00080000:2.0:1775765911.563494:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765911.563508:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775765911.563524:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775765911.563547:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2300 x1862022114398848/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00010000:00010000:2.0:1775765912.842855:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765913.610688:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:2.0:1775765914.634691:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775765914.890830:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765915.658763:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775765915.914864:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1775765916.682823:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765916.682839:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1775765916.682848:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765916.682864:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3294
00000100:00080000:2.0:1775765916.682886:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765916.682900:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775765916.682985:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1c00 x1862022114398976/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775765916.683041:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765916.683049:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775765916.683053:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765916.683063:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3294
00000100:00080000:2.0:1775765916.683071:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765916.683080:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775765916.683101:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775765916.683134:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3480 x1862022114399104/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775765916.683161:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765916.683169:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775765916.683173:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765916.683182:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3294
00000100:00080000:2.0:1775765916.683191:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765916.683199:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775765916.683214:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775765916.683248:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2300 x1862022114399232/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775765916.683264:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765916.683269:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775765916.683272:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765916.683278:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3294
00000100:00080000:2.0:1775765916.683285:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765916.683293:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775765916.683301:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775765916.683319:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3100 x1862022114399360/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775765916.683388:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765916.683394:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1775765916.683396:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765916.683404:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3294
00000100:00080000:2.0:1775765916.683408:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765916.683421:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775765916.683441:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775765916.683470:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1880 x1862022114399488/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775765916.683489:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765916.683499:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775765916.683510:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765916.683521:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3294
00000100:00080000:2.0:1775765916.683533:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765916.683541:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775765916.683547:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775765916.683606:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0e00 x1862022114399616/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775765916.683627:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765916.683635:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775765916.683639:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765916.683647:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3294
00000100:00080000:2.0:1775765916.683656:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765916.683662:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775765916.683670:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775765916.683702:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3b80 x1862022114399744/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775765916.683719:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765916.683732:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775765916.683739:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765916.683783:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3294
00000100:00080000:2.0:1775765916.683795:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765916.683816:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775765916.683826:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775765916.683851:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0000 x1862022114399872/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00010000:00010000:3.0:1775765916.938817:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00010000:00010000:2.0:1775765917.962843:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00010000:00010000:2.0:1775765920.010800:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1775765921.802753:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765921.802763:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775765921.802770:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765921.802785:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3299
00000100:00080000:3.0:1775765921.802802:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765921.802814:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775765921.802881:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747100 x1862022114400000/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765921.802912:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765921.802916:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775765921.802918:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765921.802923:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3299
00000100:00080000:3.0:1775765921.802927:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765921.802933:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765921.802946:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775765921.802969:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745f80 x1862022114400128/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765921.802982:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765921.802986:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775765921.802988:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765921.802992:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3299
00000100:00080000:3.0:1775765921.802995:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765921.802998:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765921.803003:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775765921.803032:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745180 x1862022114400256/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765921.803045:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765921.803050:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775765921.803054:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765921.803059:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3299
00000100:00080000:3.0:1775765921.803068:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765921.803080:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765921.803089:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775765921.803107:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745c00 x1862022114400384/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765921.803122:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765921.803127:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775765921.803130:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765921.803136:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3299
00000100:00080000:3.0:1775765921.803146:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765921.803169:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765921.803179:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775765921.803203:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747800 x1862022114400512/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765921.803220:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765921.803227:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775765921.803231:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765921.803240:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3299
00000100:00080000:3.0:1775765921.803245:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765921.803250:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765921.803255:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775765921.803265:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744000 x1862022114400640/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765921.803279:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765921.803288:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775765921.803289:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765921.803294:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3299
00000100:00080000:3.0:1775765921.803298:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765921.803301:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765921.803306:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775765921.803318:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745500 x1862022114400768/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765921.803343:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765921.803348:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775765921.803351:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765921.803362:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3299
00000100:00080000:3.0:1775765921.803373:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765921.803383:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765921.803408:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775765921.803437:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744700 x1862022114400896/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00010000:00010000:2.0:1775765922.058827:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00010000:00010000:3.0:1775765923.082796:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00010000:00010000:3.0:1775765924.106869:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00010000:00010000:2.0:1775765925.130852:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1775765926.922737:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765926.922753:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775765926.922759:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765926.922769:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3304
00000100:00080000:3.0:1775765926.922780:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765926.922789:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775765926.922830:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745880 x1862022114401024/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765926.922862:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765926.922866:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775765926.922869:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765926.922874:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3304
00000100:00080000:3.0:1775765926.922878:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765926.922886:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765926.922897:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775765926.922917:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746680 x1862022114401152/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765926.922943:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765926.922947:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775765926.922949:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765926.922954:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3304
00000100:00080000:3.0:1775765926.922958:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765926.922962:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765926.922967:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775765926.922983:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747b80 x1862022114401280/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765926.922996:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765926.922998:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775765926.922999:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765926.923002:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3304
00000100:00080000:3.0:1775765926.923004:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765926.923008:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765926.923012:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775765926.923023:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744e00 x1862022114401408/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765926.923047:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765926.923067:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775765926.923069:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765926.923076:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3304
00000100:00080000:3.0:1775765926.923085:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765926.923091:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765926.923095:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775765926.923111:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744380 x1862022114401536/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765926.923152:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765926.923159:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775765926.923162:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765926.923176:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3304
00000100:00080000:3.0:1775765926.923194:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765926.923202:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765926.923218:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775765926.923250:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746a00 x1862022114401664/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765926.923273:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765926.923279:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775765926.923283:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765926.923292:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3304
00000100:00080000:3.0:1775765926.923310:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765926.923318:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765926.923329:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775765926.923364:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746300 x1862022114401792/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765926.923423:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765926.923435:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775765926.923457:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765926.923468:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3304
00000100:00080000:3.0:1775765926.923496:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765926.923512:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765926.923534:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775765926.923561:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747480 x1862022114401920/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00010000:00010000:2.0:1775765927.178844:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00010000:00010000:2.0:1775765929.226859:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00010000:00010000:3.0:1775765930.250738:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00010000:00010000:3.0:1775765931.274759:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1775765932.042751:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765932.042759:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1775765932.042764:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765932.042777:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3309
00000100:00080000:1.0:1775765932.042792:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765932.042802:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775765932.042865:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fe300 x1862022114402048/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765932.042896:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765932.042900:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775765932.042902:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765932.042906:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3309
00000100:00080000:1.0:1775765932.042909:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765932.042913:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765932.042926:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775765932.042943:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff100 x1862022114402176/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765932.042962:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765932.042970:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775765932.042972:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765932.042979:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3309
00000100:00080000:1.0:1775765932.042985:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765932.042991:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765932.042999:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775765932.043018:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fea00 x1862022114402304/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765932.043031:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765932.043034:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775765932.043036:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765932.043039:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3309
00000100:00080000:1.0:1775765932.043046:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765932.043050:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765932.043055:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775765932.043077:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fc380 x1862022114402432/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765932.043092:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765932.043096:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1775765932.043099:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765932.043104:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3309
00000100:00080000:1.0:1775765932.043112:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765932.043119:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765932.043128:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775765932.043150:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fca80 x1862022114402560/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765932.043167:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765932.043171:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775765932.043173:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765932.043178:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3309
00000100:00080000:1.0:1775765932.043183:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765932.043193:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765932.043201:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775765932.043221:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fed80 x1862022114402688/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765932.043245:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765932.043260:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775765932.043263:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765932.043273:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3309
00000100:00080000:1.0:1775765932.043281:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765932.043288:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765932.043297:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775765932.043320:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff800 x1862022114402816/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765932.043351:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765932.043360:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775765932.043366:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765932.043376:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3309
00000100:00080000:1.0:1775765932.043383:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765932.043389:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765932.043398:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775765932.043430:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ffb80 x1862022114402944/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00010000:00010000:2.0:1775765932.298802:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00010000:00010000:1.0:1775765933.322832:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00010000:00010000:2.0:1775765934.346769:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000400:00000100:3.0:1775765935.114828:0:2390:0:(lib-socket.c:454:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.202.110:988
00000400:00000100:3.0:1775765935.114869:0:2390:0:(acceptor.c:111:lnet_connect_console_error()) Connection to 192.168.202.110@tcp at host 192.168.202.110 was unreachable: the network or that node may be down, or Lustre may be misconfigured.
00000800:00000100:3.0:1775765935.114888:0:2390:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:3.0:1775765935.114933:0:2390:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:3.0:1775765935.114960:0:2390:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:3.0:1775765935.114986:0:2390:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000100:00100000:2.0:1775765935.114989:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765932/real 1775765935]  req@ffff9f14ab4ff100 x1862022114402176/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765957 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000800:00000100:3.0:1775765935.115007:0:2390:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:3.0:1775765935.115029:0:2390:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000100:00100000:2.0:1775765935.115047:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff100 x1862022114402176/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765957 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000800:00000100:3.0:1775765935.115054:0:2390:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:3.0:1775765935.115077:0:2390:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000100:00080000:2.0:1775765935.115079:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775765935.115091:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775765935.115099:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114402176:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775765935.115143:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765932/real 1775765935]  req@ffff9f14ab4fea00 x1862022114402304/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765957 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775765935.115164:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fea00 x1862022114402304/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765957 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765935.115173:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775765935.115179:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775765935.115184:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114402304:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775765935.115196:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765932/real 1775765935]  req@ffff9f14ab4fc380 x1862022114402432/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765957 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775765935.115213:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fc380 x1862022114402432/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765957 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765935.115224:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775765935.115229:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775765935.115233:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114402432:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775765935.115243:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765932/real 1775765935]  req@ffff9f14ab4fe300 x1862022114402048/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765957 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775765935.115260:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fe300 x1862022114402048/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765957 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765935.115270:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775765935.115275:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775765935.115281:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114402048:192.168.202.110@tcp:250:kworker.0
00000100:00100000:2.0:1775765935.115327:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765932/real 1775765935]  req@ffff9f14ab4ffb80 x1862022114402944/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765957 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775765935.115342:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ffb80 x1862022114402944/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765957 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765935.115351:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775765935.115355:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775765935.115358:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ffb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114402944:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775765935.115366:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765932/real 1775765935]  req@ffff9f14ab4ff800 x1862022114402816/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765957 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775765935.115381:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff800 x1862022114402816/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765957 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765935.115388:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775765935.115392:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775765935.115396:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114402816:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775765935.115403:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765932/real 1775765935]  req@ffff9f14ab4fca80 x1862022114402560/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765957 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775765935.115416:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fca80 x1862022114402560/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765957 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765935.115423:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775765935.115427:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775765935.115430:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114402560:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775765935.115437:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765932/real 1775765935]  req@ffff9f14ab4fed80 x1862022114402688/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765957 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775765935.115448:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fed80 x1862022114402688/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765957 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765935.115455:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775765935.115459:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775765935.115462:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114402688:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775765936.138854:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775765936.394755:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1775765937.162656:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765937.162667:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1775765937.162672:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765937.162683:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3314
00000100:00080000:1.0:1775765937.162696:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765937.162704:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775765937.162727:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:1.0:1775765937.162750:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdc00 x1862022114403072/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765937.162783:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765937.162788:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775765937.162790:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765937.162797:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3314
00000100:00080000:1.0:1775765937.162802:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765937.162807:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775765937.162814:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114403072:192.168.202.110@tcp:250:kworker.0
00000008:00100000:1.0:1775765937.162817:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775765937.162832:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdf80 x1862022114403200/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765937.162840:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765937.162844:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775765937.162845:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765937.162849:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3314
00000100:00080000:1.0:1775765937.162852:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765937.162854:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765937.162858:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775765937.162871:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fd880 x1862022114403328/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765937.162882:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765937.162886:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775765937.162888:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765937.162893:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3314
00000100:00080000:1.0:1775765937.162897:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000800:00000100:2.0:1775765937.162900:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:1.0:1775765937.162905:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765937.162912:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775765937.162933:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fce00 x1862022114403456/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765937.162943:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765937.162947:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1775765937.162949:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765937.162954:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3314
00000100:00080000:1.0:1775765937.162959:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00100000:2.0:1775765937.162961:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765937/real 1775765937]  req@ffff9f14ab4fdc00 x1862022114403072/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765962 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765937.162966:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765937.162976:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775765937.162992:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdc00 x1862022114403072/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775765962 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765937.162999:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fed80 x1862022114403584/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775765937.163012:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765937.163017:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765937.163022:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:1.0:1775765937.163023:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775765937.163028:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:2.0:1775765937.163030:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114403072:192.168.202.110@tcp:250:kworker.0
00000100:00080000:1.0:1775765937.163036:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3314
00000100:00080000:1.0:1775765937.163042:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765937.163048:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765937.163053:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775765937.163065:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114403200:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765937.163069:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fca80 x1862022114403712/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765937.163082:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765937.163086:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775765937.163088:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765937.163093:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3314
00000100:00080000:1.0:1775765937.163099:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000800:00000100:2.0:1775765937.163102:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:1.0:1775765937.163107:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765937.163114:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775765937.163124:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765937/real 1775765937]  req@ffff9f14ab4fdf80 x1862022114403200/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765962 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775765937.163129:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff800 x1862022114403840/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765937.163151:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:2.0:1775765937.163155:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdf80 x1862022114403200/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765962 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775765937.163156:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775765937.163160:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765937.163171:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3314
00000100:00080000:2.0:1775765937.163174:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775765937.163176:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765937.163181:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00080000:2.0:1775765937.163190:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000008:00100000:1.0:1775765937.163191:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775765937.163198:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114403200:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775765937.163209:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ffb80 x1862022114403968/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775765937.163209:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114403328:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775765937.163246:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775765937.163260:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765937/real 1775765937]  req@ffff9f14ab4fd880 x1862022114403328/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765962 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775765937.163274:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fd880 x1862022114403328/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765962 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765937.163282:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775765937.163286:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775765937.163289:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114403328:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775765937.163306:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114403456:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775765937.163343:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775765937.163360:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765937/real 1775765937]  req@ffff9f14ab4fce00 x1862022114403456/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765962 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775765937.163373:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fce00 x1862022114403456/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765962 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765937.163382:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775765937.163385:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775765937.163389:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114403456:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775765937.163395:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114403584:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775765937.163417:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775765937.163428:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765937/real 1775765937]  req@ffff9f14ab4fed80 x1862022114403584/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765962 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775765937.163448:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fed80 x1862022114403584/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765962 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765937.163457:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775765937.163460:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775765937.163464:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114403584:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775765937.163485:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114403712:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775765937.163515:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775765937.163528:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765937/real 1775765937]  req@ffff9f14ab4fca80 x1862022114403712/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765962 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775765937.163542:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fca80 x1862022114403712/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765962 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765937.163549:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775765937.163597:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775765937.163602:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114403712:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775765937.163608:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114403840:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775765937.163632:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775765937.163643:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765937/real 1775765937]  req@ffff9f14ab4ff800 x1862022114403840/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765962 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775765937.163655:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff800 x1862022114403840/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775765962 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765937.163669:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775765937.163673:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775765937.163676:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114403840:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775765937.163682:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ffb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114403968:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775765937.163703:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775765937.163713:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765937/real 1775765937]  req@ffff9f14ab4ffb80 x1862022114403968/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765962 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775765937.163724:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ffb80 x1862022114403968/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775765962 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775765937.163734:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775765937.163737:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775765937.163741:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ffb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114403968:192.168.202.110@tcp:8:kworker.0
00010000:00010000:3.0:1775765937.418710:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765938.186739:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775765938.442815:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765939.210702:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775765939.466774:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765940.234679:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775765940.490765:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765941.258629:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775765941.514695:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1775765942.282793:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765942.282802:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1775765942.282808:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765942.282821:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3319
00000100:00080000:2.0:1775765942.282835:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765942.282845:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775765942.282898:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0380 x1862022114404096/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775765942.282932:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765942.282937:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775765942.282940:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765942.282945:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3319
00000100:00080000:2.0:1775765942.282951:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765942.282957:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775765942.282969:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775765942.283007:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3800 x1862022114404224/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775765942.283025:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765942.283031:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775765942.283033:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765942.283041:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3319
00000100:00080000:2.0:1775765942.283046:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765942.283051:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775765942.283057:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775765942.283075:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2d80 x1862022114404352/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775765942.283089:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765942.283093:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775765942.283096:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765942.283101:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3319
00000100:00080000:2.0:1775765942.283105:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765942.283110:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775765942.283115:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775765942.283133:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0700 x1862022114404480/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775765942.283142:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765942.283154:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1775765942.283160:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765942.283173:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3319
00000100:00080000:2.0:1775765942.283181:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765942.283186:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775765942.283191:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775765942.283208:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0a80 x1862022114404608/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775765942.283226:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765942.283235:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775765942.283239:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765942.283251:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3319
00000100:00080000:2.0:1775765942.283258:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765942.283263:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775765942.283268:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775765942.283286:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1500 x1862022114404736/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775765942.283303:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765942.283307:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775765942.283310:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765942.283315:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3319
00000100:00080000:2.0:1775765942.283321:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765942.283329:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775765942.283336:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775765942.283355:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2680 x1862022114404864/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775765942.283391:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765942.283400:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775765942.283406:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765942.283419:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3319
00000100:00080000:2.0:1775765942.283430:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765942.283437:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775765942.283443:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775765942.283466:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2a00 x1862022114404992/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00010000:00010000:2.0:1775765943.562745:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00010000:00010000:3.0:1775765944.586674:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00010000:00010000:3.0:1775765945.610755:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00010000:00010000:2.0:1775765946.634880:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1775765947.402652:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765947.402661:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775765947.402665:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765947.402675:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3324
00000100:00080000:3.0:1775765947.402686:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765947.402693:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775765947.402732:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744a80 x1862022114405120/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765947.402760:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765947.402765:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775765947.402767:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765947.402770:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3324
00000100:00080000:3.0:1775765947.402774:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765947.402778:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765947.402787:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775765947.402801:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747480 x1862022114405248/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765947.402811:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765947.402813:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775765947.402815:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765947.402817:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3324
00000100:00080000:3.0:1775765947.402820:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765947.402822:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765947.402825:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775765947.402832:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746300 x1862022114405376/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765947.402838:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765947.402841:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775765947.402842:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765947.402844:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3324
00000100:00080000:3.0:1775765947.402846:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765947.402850:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765947.402857:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775765947.402867:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746a00 x1862022114405504/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765947.402874:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765947.402876:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775765947.402877:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765947.402879:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3324
00000100:00080000:3.0:1775765947.402881:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765947.402884:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765947.402888:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775765947.402897:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744380 x1862022114405632/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765947.402916:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765947.402918:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775765947.402919:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765947.402926:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3324
00000100:00080000:3.0:1775765947.402931:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765947.402939:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765947.402943:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775765947.402955:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744e00 x1862022114405760/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765947.402969:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765947.402971:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775765947.402973:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765947.402976:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3324
00000100:00080000:3.0:1775765947.402979:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765947.402984:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765947.402991:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775765947.403001:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747b80 x1862022114405888/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765947.403008:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765947.403013:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775765947.403015:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765947.403018:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3324
00000100:00080000:3.0:1775765947.403022:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765947.403027:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765947.403033:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775765947.403045:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746680 x1862022114406016/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00010000:00010000:1.0:1775765947.658700:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00010000:00010000:2.0:1775765948.682698:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00010000:00010000:2.0:1775765950.730698:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00010000:00010000:3.0:1775765951.754714:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1775765952.522629:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765952.522637:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775765952.522641:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765952.522650:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3330
00000100:00080000:3.0:1775765952.522661:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765952.522667:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775765952.522699:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745880 x1862022114406144/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765952.522723:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765952.522725:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775765952.522726:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765952.522729:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3330
00000100:00080000:3.0:1775765952.522732:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765952.522734:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765952.522742:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775765952.522750:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744700 x1862022114406272/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765952.522762:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765952.522766:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775765952.522767:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765952.522770:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3330
00000100:00080000:3.0:1775765952.522771:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765952.522774:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765952.522777:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775765952.522784:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745500 x1862022114406400/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765952.522790:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765952.522792:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775765952.522793:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765952.522796:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3330
00000100:00080000:3.0:1775765952.522798:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765952.522801:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765952.522805:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775765952.522812:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744000 x1862022114406528/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765952.522822:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765952.522825:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775765952.522827:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765952.522830:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3330
00000100:00080000:3.0:1775765952.522833:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765952.522836:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765952.522838:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775765952.522845:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747800 x1862022114406656/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765952.522852:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765952.522856:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775765952.522859:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765952.522867:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3330
00000100:00080000:3.0:1775765952.522871:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765952.522877:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765952.522883:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775765952.522896:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745c00 x1862022114406784/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765952.522936:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765952.522942:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775765952.522945:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765952.522953:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3330
00000100:00080000:3.0:1775765952.522955:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765952.522958:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765952.522961:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775765952.522968:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745180 x1862022114406912/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765952.522984:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765952.522987:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775765952.522989:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765952.522992:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3330
00000100:00080000:3.0:1775765952.522995:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765952.523000:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765952.523004:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775765952.523032:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745f80 x1862022114407040/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00010000:00010000:3.0:1775765952.778706:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00010000:00010000:2.0:1775765953.802768:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00010000:00010000:1.0:1775765954.826715:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00010000:00010000:2.0:1775765955.850828:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1775765957.642660:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765957.642669:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775765957.642674:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765957.642691:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3335
00000100:00080000:3.0:1775765957.642704:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765957.642711:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775765957.642757:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747100 x1862022114407168/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765957.642785:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765957.642788:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775765957.642789:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765957.642792:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3335
00000100:00080000:3.0:1775765957.642795:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765957.642798:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765957.642809:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775765957.642824:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745f80 x1862022114407296/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765957.642846:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765957.642849:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775765957.642851:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765957.642855:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3335
00000100:00080000:3.0:1775765957.642859:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765957.642861:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765957.642865:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775765957.642877:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745180 x1862022114407424/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765957.642883:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765957.642886:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775765957.642887:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765957.642890:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3335
00000100:00080000:3.0:1775765957.642893:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765957.642895:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765957.642899:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775765957.642909:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745c00 x1862022114407552/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765957.642918:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765957.642920:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775765957.642922:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765957.642927:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3335
00000100:00080000:3.0:1775765957.642929:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765957.642933:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765957.642937:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775765957.642949:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747800 x1862022114407680/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765957.643011:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765957.643015:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775765957.643017:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765957.643029:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3335
00000100:00080000:3.0:1775765957.643038:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765957.643051:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765957.643069:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775765957.643089:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744000 x1862022114407808/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765957.643102:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765957.643105:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775765957.643107:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765957.643117:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3335
00000100:00080000:3.0:1775765957.643122:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765957.643129:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765957.643141:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775765957.643165:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745500 x1862022114407936/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765957.643222:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765957.643231:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775765957.643235:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765957.643254:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3335
00000100:00080000:3.0:1775765957.643265:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765957.643271:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765957.643279:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775765957.643299:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744700 x1862022114408064/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00010000:00010000:2.0:1775765957.898747:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00010000:00010000:3.0:1775765958.922730:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00010000:00010000:3.0:1775765959.946683:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00010000:00010000:2.0:1775765960.970755:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00010000:00010000:1.0:1775765961.994690:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1775765962.762702:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765962.762731:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775765962.762738:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765962.762754:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3340
00000100:00080000:3.0:1775765962.762772:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765962.762782:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775765962.762846:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745880 x1862022114408192/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765962.762882:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765962.762886:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775765962.762888:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765962.762893:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3340
00000100:00080000:3.0:1775765962.762896:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765962.762900:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765962.762925:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775765962.762941:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746680 x1862022114408320/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765962.762984:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765962.762988:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775765962.762989:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765962.763002:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3340
00000100:00080000:3.0:1775765962.763005:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765962.763009:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765962.763014:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775765962.763047:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747b80 x1862022114408448/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765962.763057:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765962.763061:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775765962.763064:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765962.763067:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3340
00000100:00080000:3.0:1775765962.763071:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765962.763075:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765962.763082:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775765962.763092:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744e00 x1862022114408576/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765962.763102:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765962.763105:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775765962.763106:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765962.763110:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3340
00000100:00080000:3.0:1775765962.763114:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765962.763122:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765962.763129:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775765962.763144:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744380 x1862022114408704/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765962.763166:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765962.763170:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775765962.763171:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765962.763176:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3340
00000100:00080000:3.0:1775765962.763180:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765962.763185:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765962.763191:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775765962.763208:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746a00 x1862022114408832/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765962.763274:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765962.763278:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775765962.763280:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765962.763285:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3340
00000100:00080000:3.0:1775765962.763289:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765962.763298:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765962.763308:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775765962.763351:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746300 x1862022114408960/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765962.763394:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765962.763400:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775765962.763404:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765962.763410:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3340
00000100:00080000:3.0:1775765962.763414:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765962.763419:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765962.763424:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775765962.763462:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747480 x1862022114409088/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00010000:00010000:2.0:1775765963.018704:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765963.786685:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:2.0:1775765964.810695:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775765965.066730:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765965.834625:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775765966.090702:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765966.858663:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775765967.114700:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765967.882626:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775765967.882626:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765967.882635:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775765967.882638:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765967.882649:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3345
00000100:00080000:3.0:1775765967.882659:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765967.882665:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775765967.882696:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744a80 x1862022114409216/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765967.882722:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765967.882726:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775765967.882728:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765967.882732:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3345
00000100:00080000:3.0:1775765967.882737:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765967.882741:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765967.882750:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775765967.882765:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747480 x1862022114409344/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765967.882774:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765967.882777:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775765967.882778:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765967.882781:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3345
00000100:00080000:3.0:1775765967.882786:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765967.882789:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765967.882793:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775765967.882803:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746300 x1862022114409472/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765967.882813:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765967.882816:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775765967.882816:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765967.882819:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3345
00000100:00080000:3.0:1775765967.882821:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765967.882823:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765967.882825:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775765967.882834:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746a00 x1862022114409600/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765967.882840:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765967.882843:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775765967.882844:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765967.882846:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3345
00000100:00080000:3.0:1775765967.882848:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765967.882850:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765967.882853:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775765967.882859:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744380 x1862022114409728/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765967.882876:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765967.882878:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775765967.882880:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765967.882884:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3345
00000100:00080000:3.0:1775765967.882886:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765967.882888:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765967.882891:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775765967.882898:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744e00 x1862022114409856/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765967.882904:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765967.882907:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775765967.882909:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765967.882913:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3345
00000100:00080000:3.0:1775765967.882916:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765967.882920:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765967.882926:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775765967.882936:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747b80 x1862022114409984/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765967.882942:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765967.882943:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775765967.882944:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765967.882947:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3345
00000100:00080000:3.0:1775765967.882950:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765967.882956:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765967.882965:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775765967.882981:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746680 x1862022114410112/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00010000:00010000:2.0:1775765968.138742:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765968.906638:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775765969.162719:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765969.930614:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775765970.186668:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765970.954630:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:2.0:1775765971.978625:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775765972.234760:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1775765973.002636:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:2.0:1775765973.002643:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775765973.002652:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775765973.002656:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765973.002664:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3350
00000100:00080000:3.0:1775765973.002674:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765973.002679:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775765973.002712:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745880 x1862022114410240/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765973.002735:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765973.002737:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775765973.002739:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765973.002742:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3350
00000100:00080000:3.0:1775765973.002744:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765973.002746:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765973.002753:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775765973.002760:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744700 x1862022114410368/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765973.002766:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765973.002770:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775765973.002771:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765973.002774:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3350
00000100:00080000:3.0:1775765973.002776:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765973.002785:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765973.002787:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775765973.002799:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745500 x1862022114410496/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765973.002806:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765973.002808:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775765973.002809:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765973.002811:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3350
00000100:00080000:3.0:1775765973.002814:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765973.002817:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765973.002821:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775765973.002829:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744000 x1862022114410624/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765973.002843:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765973.002846:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775765973.002847:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765973.002850:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3350
00000100:00080000:3.0:1775765973.002852:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765973.002854:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765973.002859:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775765973.002868:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747800 x1862022114410752/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765973.002874:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765973.002877:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775765973.002878:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765973.002881:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3350
00000100:00080000:3.0:1775765973.002883:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765973.002886:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765973.002889:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775765973.002897:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745c00 x1862022114410880/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765973.002902:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765973.002904:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775765973.002905:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765973.002908:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3350
00000100:00080000:3.0:1775765973.002911:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765973.002914:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765973.002916:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775765973.002922:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745180 x1862022114411008/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765973.002942:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765973.002945:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775765973.002947:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765973.002951:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3350
00000100:00080000:3.0:1775765973.002956:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765973.002960:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765973.002963:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775765973.002972:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745f80 x1862022114411136/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00010000:00010000:3.0:1775765973.258723:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765974.026644:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775765974.282725:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765975.050628:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775765975.306663:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765976.074657:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775765976.330687:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765977.098632:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775765977.354751:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765978.122720:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:2.0:1775765978.122817:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765978.122825:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1775765978.122830:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765978.122846:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3355
00000100:00080000:2.0:1775765978.122861:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765978.122872:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775765978.122921:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1f80 x1862022114411264/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775765978.122956:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765978.122961:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775765978.122964:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765978.122971:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3355
00000100:00080000:2.0:1775765978.122978:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765978.122985:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775765978.122999:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775765978.123027:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1180 x1862022114411392/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775765978.123072:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765978.123078:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775765978.123080:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765978.123089:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3355
00000100:00080000:2.0:1775765978.123095:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765978.123101:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775765978.123108:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775765978.123124:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2a00 x1862022114411520/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775765978.123140:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765978.123158:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775765978.123160:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765978.123191:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3355
00000100:00080000:2.0:1775765978.123199:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765978.123205:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775765978.123213:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775765978.123237:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2680 x1862022114411648/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775765978.123250:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765978.123255:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1775765978.123262:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765978.123277:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3355
00000100:00080000:2.0:1775765978.123290:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765978.123303:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775765978.123313:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775765978.123335:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1500 x1862022114411776/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775765978.123351:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765978.123366:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775765978.123386:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765978.123398:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3355
00000100:00080000:2.0:1775765978.123406:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765978.123420:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775765978.123432:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775765978.123460:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0a80 x1862022114411904/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775765978.123475:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765978.123480:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775765978.123484:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765978.123492:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3355
00000100:00080000:2.0:1775765978.123498:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765978.123503:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775765978.123512:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775765978.123531:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0700 x1862022114412032/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775765978.123606:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775765978.123610:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775765978.123614:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775765978.123621:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3355
00000100:00080000:2.0:1775765978.123629:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775765978.123633:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775765978.123641:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775765978.123664:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2d80 x1862022114412160/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775765979.146657:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775765979.402731:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765980.170712:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775765980.426782:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765981.194723:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775765981.450766:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765982.218727:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775765982.474781:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765983.242663:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:1.0:1775765983.242680:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765983.242690:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1775765983.242695:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765983.242706:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3360
00000100:00080000:1.0:1775765983.242717:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765983.242725:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775765983.242769:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fe300 x1862022114412288/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765983.242799:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765983.242803:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775765983.242805:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765983.242810:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3360
00000100:00080000:1.0:1775765983.242814:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765983.242818:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765983.242828:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775765983.242842:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fc380 x1862022114412416/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765983.242853:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765983.242856:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775765983.242858:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765983.242861:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3360
00000100:00080000:1.0:1775765983.242865:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765983.242868:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765983.242872:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775765983.242880:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fea00 x1862022114412544/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765983.242888:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765983.242891:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775765983.242893:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765983.242896:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3360
00000100:00080000:1.0:1775765983.242899:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765983.242902:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765983.242906:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775765983.242914:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff100 x1862022114412672/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765983.242923:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765983.242925:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1775765983.242927:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765983.242930:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3360
00000100:00080000:1.0:1775765983.242933:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765983.242937:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765983.242941:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775765983.242949:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ffb80 x1862022114412800/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765983.242958:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765983.242961:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775765983.242963:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765983.242966:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3360
00000100:00080000:1.0:1775765983.242970:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765983.242974:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765983.242978:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775765983.242987:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff800 x1862022114412928/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765983.242994:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765983.242997:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775765983.242998:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765983.243001:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3360
00000100:00080000:1.0:1775765983.243004:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765983.243008:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765983.243012:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775765983.243021:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fca80 x1862022114413056/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765983.243031:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765983.243034:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775765983.243036:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765983.243040:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3360
00000100:00080000:1.0:1775765983.243043:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765983.243046:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765983.243051:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775765983.243070:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fed80 x1862022114413184/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00010000:00010000:1.0:1775765983.498763:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765984.266668:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775765984.522768:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765985.290650:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:2.0:1775765986.314705:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775765986.570748:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765987.338630:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775765987.594775:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1775765988.362668:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765988.362678:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00100000:2.0:1775765988.362680:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775765988.362683:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765988.362696:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3365
00000100:00080000:3.0:1775765988.362710:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765988.362718:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775765988.362765:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747100 x1862022114413312/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765988.362793:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765988.362796:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775765988.362797:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765988.362800:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3365
00000100:00080000:3.0:1775765988.362803:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765988.362805:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765988.362816:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775765988.362829:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745f80 x1862022114413440/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765988.362837:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765988.362843:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775765988.362845:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765988.362853:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3365
00000100:00080000:3.0:1775765988.362855:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765988.362858:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765988.362861:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775765988.362873:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745180 x1862022114413568/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765988.362882:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765988.362886:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775765988.362888:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765988.362892:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3365
00000100:00080000:3.0:1775765988.362897:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765988.362901:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765988.362907:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775765988.362922:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745c00 x1862022114413696/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765988.362934:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765988.362937:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775765988.362939:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765988.362943:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3365
00000100:00080000:3.0:1775765988.362947:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765988.362950:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765988.362954:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775765988.362965:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747800 x1862022114413824/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765988.362989:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765988.362992:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775765988.362993:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765988.362997:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3365
00000100:00080000:3.0:1775765988.363001:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765988.363009:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765988.363016:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775765988.363026:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744000 x1862022114413952/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765988.363040:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765988.363043:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775765988.363045:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765988.363049:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3365
00000100:00080000:3.0:1775765988.363052:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765988.363055:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765988.363060:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775765988.363092:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745500 x1862022114414080/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765988.363112:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765988.363119:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775765988.363124:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765988.363132:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3365
00000100:00080000:3.0:1775765988.363139:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765988.363148:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765988.363163:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775765988.363191:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744700 x1862022114414208/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00010000:00010000:1.0:1775765988.618814:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765989.386671:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775765989.642775:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765990.410660:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775765990.666719:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765991.434666:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775765991.690720:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765992.458724:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775765993.482700:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765993.482711:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775765993.482715:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00100000:2.0:1775765993.482723:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775765993.482728:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3370
00000100:00080000:3.0:1775765993.482745:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765993.482753:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775765993.482808:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745880 x1862022114414336/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765993.482834:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765993.482837:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775765993.482840:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765993.482844:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3370
00000100:00080000:3.0:1775765993.482849:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765993.482853:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765993.482864:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775765993.482881:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746680 x1862022114414464/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765993.482905:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765993.482913:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775765993.482915:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765993.482920:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3370
00000100:00080000:3.0:1775765993.482923:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765993.482928:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765993.482934:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775765993.482946:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747b80 x1862022114414592/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765993.482954:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765993.482957:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775765993.482959:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765993.482962:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3370
00000100:00080000:3.0:1775765993.482968:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765993.482970:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765993.482973:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775765993.482985:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744e00 x1862022114414720/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765993.482991:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765993.482993:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775765993.482995:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765993.483005:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3370
00000100:00080000:3.0:1775765993.483013:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765993.483018:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765993.483024:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775765993.483061:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744380 x1862022114414848/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765993.483070:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765993.483072:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775765993.483074:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765993.483076:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3370
00000100:00080000:3.0:1775765993.483080:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765993.483083:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765993.483090:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775765993.483108:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746a00 x1862022114414976/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765993.483187:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765993.483193:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775765993.483201:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765993.483216:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3370
00000100:00080000:3.0:1775765993.483227:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765993.483231:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765993.483238:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775765993.483260:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746300 x1862022114415104/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775765993.483267:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775765993.483270:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775765993.483271:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775765993.483274:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3370
00000100:00080000:3.0:1775765993.483276:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775765993.483278:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775765993.483285:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775765993.483298:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747480 x1862022114415232/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00010000:00010000:1.0:1775765993.738771:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765994.506633:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775765994.762766:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765995.530639:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775765995.786861:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765996.554625:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775765996.810739:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765997.578645:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775765997.834905:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765998.602659:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:1.0:1775765998.602727:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765998.602735:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1775765998.602739:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765998.602748:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3376
00000100:00080000:1.0:1775765998.602760:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765998.602767:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775765998.602803:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fce00 x1862022114415360/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765998.602827:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765998.602829:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775765998.602831:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765998.602834:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3376
00000100:00080000:1.0:1775765998.602837:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765998.602839:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765998.602848:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775765998.602863:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fd880 x1862022114415488/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765998.602875:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765998.602879:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775765998.602880:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765998.602884:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3376
00000100:00080000:1.0:1775765998.602890:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765998.602907:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765998.602920:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775765998.602939:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdf80 x1862022114415616/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765998.602948:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765998.602951:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775765998.602952:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765998.602956:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3376
00000100:00080000:1.0:1775765998.602960:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765998.602963:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765998.602969:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775765998.602992:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdc00 x1862022114415744/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765998.603002:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765998.603005:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1775765998.603007:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765998.603012:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3376
00000100:00080000:1.0:1775765998.603018:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765998.603022:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765998.603028:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775765998.603043:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fed80 x1862022114415872/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765998.603052:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765998.603056:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775765998.603058:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765998.603068:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3376
00000100:00080000:1.0:1775765998.603076:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765998.603082:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765998.603090:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775765998.603100:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fca80 x1862022114416000/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765998.603107:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765998.603110:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775765998.603113:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765998.603119:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3376
00000100:00080000:1.0:1775765998.603124:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765998.603131:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765998.603135:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775765998.603146:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff800 x1862022114416128/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775765998.603155:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775765998.603158:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775765998.603160:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775765998.603163:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3376
00000100:00080000:1.0:1775765998.603166:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775765998.603169:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775765998.603178:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775765998.603192:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ffb80 x1862022114416256/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00010000:00010000:2.0:1775765998.858889:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775765999.626720:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:2.0:1775766000.650657:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775766000.906850:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766001.674805:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:2.0:1775766001.675506:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765998/real 1775766001]  req@ffff9f14ab4ff800 x1862022114416128/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766023 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766001.675546:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff800 x1862022114416128/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766023 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766001.675615:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766001.675623:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766001.675628:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114416128:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766001.675637:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765998/real 1775766001]  req@ffff9f14ab4ffb80 x1862022114416256/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766023 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766001.675653:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ffb80 x1862022114416256/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766023 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766001.675662:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766001.675666:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766001.675670:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ffb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114416256:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775766001.675677:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765998/real 1775766001]  req@ffff9f14ab4fdf80 x1862022114415616/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766023 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766001.675690:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdf80 x1862022114415616/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766023 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766001.675699:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766001.675703:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766001.675707:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114415616:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766001.675716:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765998/real 1775766001]  req@ffff9f14ab4fdc00 x1862022114415744/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766023 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766001.675730:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdc00 x1862022114415744/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766023 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766001.675739:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766001.675744:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766001.675748:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114415744:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775766001.675755:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765998/real 1775766001]  req@ffff9f14ab4fed80 x1862022114415872/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766023 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766001.675767:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fed80 x1862022114415872/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766023 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766001.675775:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766001.675778:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766001.675782:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114415872:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775766001.675790:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765998/real 1775766001]  req@ffff9f14ab4fca80 x1862022114416000/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766023 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766001.675803:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fca80 x1862022114416000/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766023 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766001.675812:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766001.675815:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766001.675830:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114416000:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766001.675838:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765998/real 1775766001]  req@ffff9f14ab4fce00 x1862022114415360/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766023 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766001.675852:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fce00 x1862022114415360/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766023 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766001.675861:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766001.675865:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766001.675868:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114415360:192.168.202.110@tcp:250:kworker.0
00000100:00100000:2.0:1775766001.675875:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775765998/real 1775766001]  req@ffff9f14ab4fd880 x1862022114415488/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766023 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766001.675888:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fd880 x1862022114415488/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766023 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766001.675897:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766001.675901:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766001.675904:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114415488:192.168.202.110@tcp:38:kworker.0
00010000:00010000:3.0:1775766001.930758:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766002.698684:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775766002.954703:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1775766003.722622:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766003.722630:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1775766003.722633:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766003.722641:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3381
00000100:00080000:1.0:1775766003.722651:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766003.722657:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775766003.722674:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:1.0:1775766003.722688:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff100 x1862022114416384/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766003.722709:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766003.722711:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775766003.722713:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766003.722715:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3381
00000100:00080000:1.0:1775766003.722718:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766003.722720:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766003.722728:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775766003.722737:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fea00 x1862022114416512/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766003.722744:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:2.0:1775766003.722747:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114416384:192.168.202.110@tcp:250:kworker.0
00000100:00080000:1.0:1775766003.722748:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775766003.722751:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766003.722760:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3381
00000100:00080000:1.0:1775766003.722764:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766003.722768:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766003.722773:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775766003.722784:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fc380 x1862022114416640/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766003.722794:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766003.722798:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775766003.722800:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766003.722805:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3381
00000100:00080000:1.0:1775766003.722810:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766003.722815:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766003.722820:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:2.0:1775766003.722820:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775766003.722834:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fe300 x1862022114416768/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766003.722845:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766003.722849:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1775766003.722850:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766003.722855:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3381
00000100:00080000:1.0:1775766003.722859:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766003.722863:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775766003.722868:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766003/real 1775766003]  req@ffff9f14ab4ff100 x1862022114416384/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766028 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:1.0:1775766003.722871:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775766003.722887:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fd880 x1862022114416896/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775766003.722896:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff100 x1862022114416384/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766028 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766003.722899:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766003.722906:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775766003.722910:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766003.722919:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3381
00000100:00080000:2.0:1775766003.722920:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775766003.722924:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766003.722942:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:1.0:1775766003.722948:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775766003.722951:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114416384:192.168.202.110@tcp:250:kworker.0
00000008:00100000:1.0:1775766003.722956:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775766003.722971:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fce00 x1862022114417024/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775766003.722981:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114416512:192.168.202.110@tcp:38:kworker.0
00000100:00080000:1.0:1775766003.722982:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766003.722990:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775766003.722993:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766003.722997:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3381
00000100:00080000:1.0:1775766003.723001:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766003.723005:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766003.723010:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775766003.723025:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fca80 x1862022114417152/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:2.0:1775766003.723025:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:1.0:1775766003.723037:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766003.723041:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775766003.723043:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:2.0:1775766003.723046:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766003/real 1775766003]  req@ffff9f14ab4fea00 x1862022114416512/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766028 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766003.723050:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3381
00000100:00080000:1.0:1775766003.723056:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766003.723064:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766003.723070:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775766003.723073:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fea00 x1862022114416512/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766028 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766003.723083:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00100000:1.0:1775766003.723084:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fed80 x1862022114417280/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766003.723091:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766003.723099:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114416512:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766003.723108:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114416640:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775766003.723131:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766003.723142:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766003/real 1775766003]  req@ffff9f14ab4fc380 x1862022114416640/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766028 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766003.723154:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fc380 x1862022114416640/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766028 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766003.723162:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766003.723166:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766003.723169:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114416640:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766003.723176:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114416768:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775766003.723196:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766003.723207:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766003/real 1775766003]  req@ffff9f14ab4fe300 x1862022114416768/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766028 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766003.723219:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fe300 x1862022114416768/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766028 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766003.723228:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766003.723231:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766003.723234:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114416768:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775766003.723240:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114416896:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775766003.723259:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766003.723270:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766003/real 1775766003]  req@ffff9f14ab4fd880 x1862022114416896/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766028 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766003.723281:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fd880 x1862022114416896/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766028 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766003.723289:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766003.723292:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766003.723296:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114416896:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775766003.723314:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114417024:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775766003.723332:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766003.723343:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766003/real 1775766003]  req@ffff9f14ab4fce00 x1862022114417024/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766028 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766003.723354:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fce00 x1862022114417024/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766028 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766003.723362:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766003.723365:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766003.723368:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114417024:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766003.723374:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114417152:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775766003.723391:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766003.723402:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766003/real 1775766003]  req@ffff9f14ab4fca80 x1862022114417152/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766028 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766003.723414:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fca80 x1862022114417152/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766028 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766003.723421:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766003.723424:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766003.723427:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114417152:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766003.723434:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114417280:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775766003.723453:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766003.723464:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766003/real 1775766003]  req@ffff9f14ab4fed80 x1862022114417280/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766028 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766003.723475:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fed80 x1862022114417280/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766028 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766003.723483:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766003.723486:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766003.723490:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114417280:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775766003.978720:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766004.746779:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766005.002798:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766005.770680:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766006.026771:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766006.794702:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:2.0:1775766007.818625:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766008.074718:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1775766008.842728:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766008.842741:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1775766008.842746:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766008.842757:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3386
00000100:00100000:2.0:1775766008.842766:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:1.0:1775766008.842770:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766008.842781:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775766008.842827:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdc00 x1862022114417408/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766008.842862:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766008.842867:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775766008.842869:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766008.842873:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3386
00000100:00080000:1.0:1775766008.842877:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766008.842915:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766008.842927:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775766008.842949:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdf80 x1862022114417536/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775766008.842950:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114417408:192.168.202.110@tcp:250:kworker.0
00000100:00080000:1.0:1775766008.842974:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766008.842979:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775766008.842981:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766008.842985:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3386
00000100:00080000:1.0:1775766008.842989:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766008.842993:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766008.842998:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775766008.843012:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ffb80 x1862022114417664/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766008.843022:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766008.843026:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775766008.843028:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766008.843033:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3386
00000100:00080000:1.0:1775766008.843037:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766008.843040:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766008.843046:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:2.0:1775766008.843054:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775766008.843061:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff800 x1862022114417792/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766008.843073:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766008.843077:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1775766008.843079:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766008.843084:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3386
00000100:00080000:1.0:1775766008.843088:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766008.843091:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766008.843098:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775766008.843112:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fed80 x1862022114417920/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775766008.843123:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766008/real 1775766008]  req@ffff9f14ab4fdc00 x1862022114417408/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766033 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766008.843126:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766008.843135:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775766008.843140:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766008.843150:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3386
00000100:00080000:1.0:1775766008.843157:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766008.843165:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766008.843170:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766008.843177:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdc00 x1862022114417408/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766033 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775766008.843183:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fca80 x1862022114418048/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766008.843221:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766008.843224:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775766008.843224:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775766008.843230:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766008.843239:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3386
00000100:00080000:2.0:1775766008.843239:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:1.0:1775766008.843246:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766008.843254:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775766008.843255:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114417408:192.168.202.110@tcp:250:kworker.0
00000008:00100000:1.0:1775766008.843258:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775766008.843276:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fce00 x1862022114418176/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766008.843286:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766008.843289:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775766008.843292:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766008.843297:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3386
00000100:00080000:1.0:1775766008.843301:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:2.0:1775766008.843308:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114417536:192.168.202.110@tcp:38:kworker.0
00000100:00080000:1.0:1775766008.843310:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766008.843322:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775766008.843336:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fd880 x1862022114418304/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:2.0:1775766008.843368:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766008.843396:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766008/real 1775766008]  req@ffff9f14ab4fdf80 x1862022114417536/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766033 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766008.843419:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdf80 x1862022114417536/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766033 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766008.843434:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766008.843440:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766008.843446:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114417536:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766008.843456:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ffb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114417664:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775766008.843488:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766008.843505:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766008/real 1775766008]  req@ffff9f14ab4ffb80 x1862022114417664/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766033 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766008.843526:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ffb80 x1862022114417664/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766033 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766008.843539:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766008.843544:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766008.843550:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ffb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114417664:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766008.843598:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114417792:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775766008.843635:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766008.843653:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766008/real 1775766008]  req@ffff9f14ab4ff800 x1862022114417792/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766033 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766008.843675:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff800 x1862022114417792/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766033 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766008.843689:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766008.843695:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766008.843700:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114417792:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775766008.843710:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114417920:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775766008.843742:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766008.843761:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766008/real 1775766008]  req@ffff9f14ab4fed80 x1862022114417920/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766033 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766008.843782:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fed80 x1862022114417920/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766033 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766008.843795:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766008.843800:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766008.843805:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114417920:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775766008.843815:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114418048:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775766008.843846:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766008.843863:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766008/real 1775766008]  req@ffff9f14ab4fca80 x1862022114418048/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766033 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766008.843904:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fca80 x1862022114418048/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766033 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766008.843919:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766008.843924:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766008.843929:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114418048:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766008.843939:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114418176:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775766008.843971:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766008.843988:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766008/real 1775766008]  req@ffff9f14ab4fce00 x1862022114418176/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766033 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766008.844008:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fce00 x1862022114418176/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766033 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766008.844021:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766008.844026:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766008.844032:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114418176:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766008.844063:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114418304:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775766008.844095:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766008.844113:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766008/real 1775766008]  req@ffff9f14ab4fd880 x1862022114418304/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766033 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766008.844134:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fd880 x1862022114418304/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766033 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766008.844149:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766008.844154:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766008.844160:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114418304:192.168.202.110@tcp:8:kworker.0
00010000:00010000:3.0:1775766009.098825:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766009.866649:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775766010.122801:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766010.890690:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766011.146720:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766011.914652:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766012.170684:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766012.938657:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766013.194677:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1775766013.962648:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:2.0:1775766013.962655:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:1.0:1775766013.962656:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1775766013.962660:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766013.962670:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3391
00000100:00080000:1.0:1775766013.962680:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766013.962686:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775766013.962748:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fe300 x1862022114418432/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766013.962793:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766013.962798:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775766013.962800:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766013.962807:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3391
00000100:00080000:1.0:1775766013.962812:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766013.962816:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766013.962824:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775766013.962851:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fc380 x1862022114418560/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766013.962866:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766013.962895:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775766013.962897:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766013.962900:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3391
00000100:00080000:1.0:1775766013.962902:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766013.962906:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766013.962910:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766013.962920:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114418432:192.168.202.110@tcp:250:kworker.0
00000100:00100000:1.0:1775766013.962922:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fea00 x1862022114418688/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766013.962946:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766013.962949:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775766013.962950:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766013.962953:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3391
00000100:00080000:1.0:1775766013.962956:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766013.962959:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766013.962964:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775766013.962976:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff100 x1862022114418816/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766013.962983:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766013.962987:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1775766013.962988:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766013.962992:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3391
00000100:00080000:1.0:1775766013.962995:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766013.962998:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766013.963005:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775766013.963013:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fd880 x1862022114418944/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766013.963020:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766013.963024:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775766013.963026:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766013.963030:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3391
00000100:00080000:1.0:1775766013.963033:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766013.963037:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766013.963040:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000800:00000100:3.0:1775766013.963048:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775766013.963060:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fce00 x1862022114419072/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766013.963080:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766013.963083:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775766013.963084:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766013.963089:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3391
00000100:00080000:1.0:1775766013.963099:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766013.963103:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766013.963108:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775766013.963122:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fca80 x1862022114419200/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775766013.963134:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766013/real 1775766013]  req@ffff9f14ab4fe300 x1862022114418432/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766038 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766013.963136:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766013.963153:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775766013.963162:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766013.963177:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3391
00000100:00080000:1.0:1775766013.963181:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766013.963183:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766013.963188:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775766013.963196:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fed80 x1862022114419328/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775766013.963304:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fe300 x1862022114418432/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766038 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766013.963469:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766013.963476:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766013.963484:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114418432:192.168.202.110@tcp:250:kworker.0
00000100:00100000:3.0:1775766013.963508:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114418560:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775766013.963681:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766013.963713:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766013/real 1775766013]  req@ffff9f14ab4fc380 x1862022114418560/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766038 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766013.963749:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fc380 x1862022114418560/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766038 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766013.963760:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766013.963764:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766013.963768:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114418560:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766013.963817:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114418688:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775766013.963854:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766013.963865:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766013/real 1775766013]  req@ffff9f14ab4fea00 x1862022114418688/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766038 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766013.963893:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fea00 x1862022114418688/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766038 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766013.963903:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766013.963907:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766013.963910:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114418688:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766013.963917:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114418816:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775766013.963942:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766013.963953:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766013/real 1775766013]  req@ffff9f14ab4ff100 x1862022114418816/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766038 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766013.963968:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff100 x1862022114418816/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766038 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766013.963977:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766013.963981:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766013.963984:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114418816:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775766013.963990:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114418944:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775766013.964008:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766013.964018:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766013/real 1775766013]  req@ffff9f14ab4fd880 x1862022114418944/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766038 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766013.964031:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fd880 x1862022114418944/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766038 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766013.964040:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766013.964043:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766013.964046:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114418944:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775766013.964052:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114419072:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775766013.964096:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766013.964104:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766013/real 1775766013]  req@ffff9f14ab4fce00 x1862022114419072/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766038 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766013.964113:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fce00 x1862022114419072/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766038 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766013.964126:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766013.964129:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766013.964132:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114419072:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766013.964137:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114419200:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775766013.964149:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766013.964156:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766013/real 1775766013]  req@ffff9f14ab4fca80 x1862022114419200/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766038 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766013.964165:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fca80 x1862022114419200/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766038 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766013.964171:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766013.964174:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766013.964176:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114419200:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766013.964181:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114419328:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775766013.964194:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766013.964202:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766013/real 1775766013]  req@ffff9f14ab4fed80 x1862022114419328/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766038 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766013.964210:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fed80 x1862022114419328/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766038 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766013.964217:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766013.964219:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766013.964222:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114419328:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775766014.986642:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766015.242779:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766016.010639:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775766016.266747:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766017.034628:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775766017.290714:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766018.058624:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766018.314730:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1775766019.082671:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766019.082680:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00100000:2.0:1775766019.082681:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:1.0:1775766019.082684:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766019.082697:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3396
00000100:00080000:1.0:1775766019.082707:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766019.082713:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775766019.082779:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff800 x1862022114419456/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766019.082855:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766019.082859:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775766019.082861:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766019.082865:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3396
00000100:00080000:1.0:1775766019.082868:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766019.082872:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766019.082882:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775766019.082901:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ffb80 x1862022114419584/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775766019.082909:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114419456:192.168.202.110@tcp:250:kworker.0
00000100:00080000:1.0:1775766019.082915:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766019.082918:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775766019.082920:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766019.082924:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3396
00000100:00080000:1.0:1775766019.082927:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766019.082930:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766019.082934:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775766019.082946:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdf80 x1862022114419712/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766019.082953:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766019.082955:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775766019.082957:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766019.082959:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3396
00000100:00080000:1.0:1775766019.082964:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766019.082966:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766019.082969:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:3.0:1775766019.082975:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775766019.082979:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdc00 x1862022114419840/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766019.082988:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766019.082991:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1775766019.082993:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766019.082996:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3396
00000100:00080000:1.0:1775766019.082999:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766019.083002:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766019.083005:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775766019.083020:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fed80 x1862022114419968/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775766019.083021:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766019/real 1775766019]  req@ffff9f14ab4ff800 x1862022114419456/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766044 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766019.083044:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766019.083047:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775766019.083049:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766019.083053:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3396
00000100:00080000:1.0:1775766019.083056:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:3.0:1775766019.083059:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff800 x1862022114419456/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766044 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766019.083063:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766019.083069:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00080000:3.0:1775766019.083080:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00100000:1.0:1775766019.083083:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fca80 x1862022114420096/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766019.083088:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766019.083097:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114419456:192.168.202.110@tcp:250:kworker.0
00000100:00080000:1.0:1775766019.083100:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766019.083106:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775766019.083107:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766019.083111:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3396
00000100:00080000:1.0:1775766019.083113:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766019.083116:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766019.083119:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775766019.083144:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fce00 x1862022114420224/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766019.083154:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766019.083157:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775766019.083160:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:3.0:1775766019.083160:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ffb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114419584:192.168.202.110@tcp:38:kworker.0
00000100:00080000:1.0:1775766019.083165:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3396
00000100:00080000:1.0:1775766019.083170:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766019.083173:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766019.083177:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775766019.083186:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fd880 x1862022114420352/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:3.0:1775766019.083206:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766019.083243:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766019/real 1775766019]  req@ffff9f14ab4ffb80 x1862022114419584/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766044 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766019.083262:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ffb80 x1862022114419584/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766044 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766019.083270:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766019.083273:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766019.083277:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ffb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114419584:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766019.083283:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114419712:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775766019.083304:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766019.083313:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766019/real 1775766019]  req@ffff9f14ab4fdf80 x1862022114419712/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766044 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766019.083324:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdf80 x1862022114419712/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766044 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766019.083331:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766019.083334:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766019.083336:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114419712:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766019.083341:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114419840:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775766019.083355:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766019.083362:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766019/real 1775766019]  req@ffff9f14ab4fdc00 x1862022114419840/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766044 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766019.083372:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdc00 x1862022114419840/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766044 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766019.083379:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766019.083382:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766019.083384:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114419840:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775766019.083389:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114419968:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775766019.083408:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766019.083437:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766019/real 1775766019]  req@ffff9f14ab4fed80 x1862022114419968/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766044 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766019.083463:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fed80 x1862022114419968/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766044 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766019.083475:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766019.083479:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766019.083483:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114419968:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775766019.083508:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114420096:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775766019.083531:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766019.083542:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766019/real 1775766019]  req@ffff9f14ab4fca80 x1862022114420096/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766044 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766019.083552:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fca80 x1862022114420096/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766044 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766019.083558:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766019.083576:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766019.083579:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114420096:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766019.083594:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114420224:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775766019.083609:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766019.083616:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766019/real 1775766019]  req@ffff9f14ab4fce00 x1862022114420224/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766044 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766019.083624:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fce00 x1862022114420224/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766044 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766019.083698:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766019.083701:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766019.083704:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114420224:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766019.083720:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114420352:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775766019.083766:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766019.083776:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766019/real 1775766019]  req@ffff9f14ab4fd880 x1862022114420352/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766044 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766019.083821:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fd880 x1862022114420352/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766044 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766019.083833:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766019.083836:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766019.083838:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114420352:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775766019.338713:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766020.106640:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766020.362678:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766021.130665:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:2.0:1775766022.154646:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766022.410779:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766023.178639:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775766023.434926:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766024.202713:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775766024.202800:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766024.202811:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775766024.202816:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766024.202828:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3401
00000100:00080000:3.0:1775766024.202841:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766024.202851:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775766024.202898:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744a80 x1862022114420480/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766024.202949:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766024.202955:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775766024.202958:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766024.202962:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3401
00000100:00080000:3.0:1775766024.202967:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766024.202972:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766024.202986:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766024.203006:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114420480:192.168.202.110@tcp:250:kworker.0
00000100:00100000:3.0:1775766024.203009:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747480 x1862022114420608/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766024.203031:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766024.203036:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775766024.203039:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766024.203044:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3401
00000100:00080000:3.0:1775766024.203049:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766024.203054:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766024.203060:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766024.203074:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746300 x1862022114420736/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766024.203086:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766024.203091:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775766024.203093:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000800:00000100:2.0:1775766024.203100:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775766024.203108:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3401
00000100:00080000:3.0:1775766024.203116:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766024.203122:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766024.203130:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766024.203146:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746a00 x1862022114420864/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766024.203158:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:2.0:1775766024.203164:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766024/real 1775766024]  req@ffff9f1485744a80 x1862022114420480/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766049 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766024.203166:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775766024.203173:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766024.203190:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3401
00000100:00080000:3.0:1775766024.203200:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766024.203205:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775766024.203212:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744a80 x1862022114420480/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766049 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:3.0:1775766024.203213:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00080000:2.0:1775766024.203236:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00100000:3.0:1775766024.203239:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744380 x1862022114420992/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766024.203248:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766024.203262:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114420480:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775766024.203272:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766024.203278:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775766024.203282:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766024.203289:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3401
00000100:00100000:2.0:1775766024.203301:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114420608:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775766024.203306:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766024.203317:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766024.203325:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000800:00000100:2.0:1775766024.203338:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766024.203347:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744e00 x1862022114421120/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775766024.203361:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766024/real 1775766024]  req@ffff9f1485747480 x1862022114420608/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766049 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766024.203366:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766024.203381:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775766024.203386:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766024.203399:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3401
00000100:00100000:2.0:1775766024.203405:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747480 x1862022114420608/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766049 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766024.203411:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766024.203429:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00080000:2.0:1775766024.203449:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000008:00100000:3.0:1775766024.203450:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00080000:2.0:1775766024.203457:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766024.203465:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114420608:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766024.203484:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114420736:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766024.203499:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747b80 x1862022114421248/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:2.0:1775766024.203516:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766024.203534:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766024/real 1775766024]  req@ffff9f1485746300 x1862022114420736/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766049 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766024.203536:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766024.203612:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775766024.203623:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:2.0:1775766024.203637:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746300 x1862022114420736/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766049 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766024.203641:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3401
00000100:00080000:2.0:1775766024.203662:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766024.203663:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766024.203674:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00080000:3.0:1775766024.203682:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775766024.203684:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114420736:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766024.203709:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114420864:192.168.202.110@tcp:8:kworker.0
00000008:00100000:3.0:1775766024.203712:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766024.203790:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746680 x1862022114421376/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:2.0:1775766024.203799:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766024.203825:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766024/real 1775766024]  req@ffff9f1485746a00 x1862022114420864/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766049 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766024.203842:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746a00 x1862022114420864/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766049 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766024.203851:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766024.203854:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766024.203858:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114420864:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775766024.203864:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114420992:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775766024.203883:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766024.203892:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766024/real 1775766024]  req@ffff9f1485744380 x1862022114420992/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766049 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766024.203903:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744380 x1862022114420992/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766049 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766024.203911:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766024.203914:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766024.203917:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114420992:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775766024.203935:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114421120:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775766024.203950:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766024.203959:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766024/real 1775766024]  req@ffff9f1485744e00 x1862022114421120/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766049 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766024.203971:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744e00 x1862022114421120/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766049 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766024.203979:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766024.203981:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766024.203984:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114421120:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766024.203990:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114421248:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775766024.204004:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766024.204012:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766024/real 1775766024]  req@ffff9f1485747b80 x1862022114421248/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766049 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766024.204023:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747b80 x1862022114421248/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766049 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766024.204031:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766024.204033:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766024.204036:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114421248:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766024.204041:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114421376:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775766024.204055:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766024.204063:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766024/real 1775766024]  req@ffff9f1485746680 x1862022114421376/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766049 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766024.204074:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746680 x1862022114421376/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766049 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766024.204082:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766024.204084:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766024.204087:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114421376:192.168.202.110@tcp:8:kworker.0
00010000:00010000:3.0:1775766024.458930:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766025.226674:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766025.482749:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766026.250702:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766026.507743:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766027.274701:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766027.530792:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766028.298667:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:2.0:1775766029.322804:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766029.322817:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1775766029.322825:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00100000:3.0:1775766029.322828:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:2.0:1775766029.322860:0:8:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3406
00000100:00080000:2.0:1775766029.322882:0:8:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766029.322896:0:8:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775766029.322991:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3800 x1862022114421504/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766029.323067:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766029.323074:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775766029.323078:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766029.323087:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3406
00000100:00080000:2.0:1775766029.323096:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766029.323110:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775766029.323110:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114421504:192.168.202.110@tcp:250:kworker.0
00000008:00100000:2.0:1775766029.323134:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766029.323157:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0380 x1862022114421632/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766029.323172:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766029.323177:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775766029.323179:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766029.323183:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3406
00000100:00080000:2.0:1775766029.323192:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766029.323197:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766029.323202:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000800:00000100:3.0:1775766029.323215:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766029.323217:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0000 x1862022114421760/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766029.323230:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766029.323234:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775766029.323236:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766029.323241:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3406
00000100:00080000:2.0:1775766029.323245:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766029.323249:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766029.323254:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775766029.323265:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3b80 x1862022114421888/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766029.323275:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:3.0:1775766029.323281:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766029/real 1775766029]  req@ffff9f14852d3800 x1862022114421504/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766054 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766029.323282:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1775766029.323287:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766029.323300:0:8:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3406
00000100:00080000:2.0:1775766029.323308:0:8:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766029.323313:0:8:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766029.323319:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766029.323327:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3800 x1862022114421504/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766054 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766029.323334:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0e00 x1862022114422016/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766029.323364:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766029.323365:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766029.323375:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775766029.323379:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:2.0:1775766029.323380:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766029.323392:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3406
00000100:00100000:3.0:1775766029.323392:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114421504:192.168.202.110@tcp:250:kworker.0
00000100:00080000:2.0:1775766029.323404:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766029.323414:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766029.323422:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766029.323433:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114421632:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766029.323446:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1880 x1862022114422144/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:3.0:1775766029.323466:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:2.0:1775766029.323468:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766029.323475:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775766029.323480:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:3.0:1775766029.323485:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766029/real 1775766029]  req@ffff9f14852d0380 x1862022114421632/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766054 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766029.323494:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3406
00000100:00080000:2.0:1775766029.323511:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:3.0:1775766029.323516:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0380 x1862022114421632/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766054 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766029.323526:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00080000:3.0:1775766029.323538:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000008:00100000:2.0:1775766029.323539:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00080000:3.0:1775766029.323548:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766029.323587:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114421632:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766029.323618:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114421760:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766029.323648:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3100 x1862022114422272/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766029.323670:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000800:00000100:3.0:1775766029.323681:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:2.0:1775766029.323705:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775766029.323712:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766029.323740:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3406
00000100:00080000:2.0:1775766029.323749:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:3.0:1775766029.323751:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766029/real 1775766029]  req@ffff9f14852d0000 x1862022114421760/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766054 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766029.323763:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766029.323778:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766029.323785:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0000 x1862022114421760/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766054 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766029.323798:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00100000:2.0:1775766029.323802:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2300 x1862022114422400/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766029.323808:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766029.323818:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114421760:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766029.323836:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114421888:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775766029.323873:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766029.323886:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766029/real 1775766029]  req@ffff9f14852d3b80 x1862022114421888/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766054 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766029.323899:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3b80 x1862022114421888/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766054 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766029.323908:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766029.323914:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766029.323917:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114421888:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775766029.323923:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114422016:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775766029.323939:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766029.323947:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766029/real 1775766029]  req@ffff9f14852d0e00 x1862022114422016/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766054 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766029.323959:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0e00 x1862022114422016/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766054 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766029.323966:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766029.323969:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766029.323971:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114422016:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775766029.323991:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114422144:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775766029.324006:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766029.324015:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766029/real 1775766029]  req@ffff9f14852d1880 x1862022114422144/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766054 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766029.324028:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1880 x1862022114422144/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766054 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766029.324036:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766029.324039:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766029.324042:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114422144:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766029.324047:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114422272:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775766029.324062:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766029.324069:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766029/real 1775766029]  req@ffff9f14852d3100 x1862022114422272/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766054 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766029.324081:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3100 x1862022114422272/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766054 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766029.324089:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766029.324092:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766029.324094:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114422272:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766029.324099:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114422400:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775766029.324113:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766029.324120:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766029/real 1775766029]  req@ffff9f14852d2300 x1862022114422400/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766054 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766029.324132:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2300 x1862022114422400/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766054 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766029.324156:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766029.324159:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766029.324162:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114422400:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775766029.578917:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775766030.346697:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775766030.602731:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775766031.370642:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775766031.626702:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775766032.394698:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766032.650758:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775766033.418631:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766033.674710:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1775766034.442768:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:3.0:1775766034.442781:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:2.0:1775766034.442783:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1775766034.442794:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766034.442808:0:8:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3411
00000100:00080000:2.0:1775766034.442825:0:8:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766034.442835:0:8:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775766034.442887:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3480 x1862022114422528/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766034.442928:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766034.442935:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775766034.442938:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766034.442945:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3411
00000100:00080000:2.0:1775766034.442953:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766034.442963:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766034.442978:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766034.443001:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114422528:192.168.202.110@tcp:250:kworker.0
00000100:00100000:2.0:1775766034.443029:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1c00 x1862022114422656/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766034.443049:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766034.443054:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775766034.443057:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766034.443063:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3411
00000100:00080000:2.0:1775766034.443070:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766034.443076:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766034.443085:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000800:00000100:3.0:1775766034.443092:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766034.443111:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2300 x1862022114422784/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766034.443127:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766034.443131:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775766034.443133:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766034.443155:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3411
00000100:00080000:2.0:1775766034.443160:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766034.443164:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766034.443171:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766034.443183:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766034/real 1775766034]  req@ffff9f14852d3480 x1862022114422528/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766059 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766034.443192:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3100 x1862022114422912/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766034.443233:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:3.0:1775766034.443240:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3480 x1862022114422528/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766059 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766034.443243:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1775766034.443251:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766034.443266:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766034.443267:0:8:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3411
00000100:00080000:3.0:1775766034.443278:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:2.0:1775766034.443282:0:8:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00100000:3.0:1775766034.443284:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114422528:192.168.202.110@tcp:250:kworker.0
00000100:00080000:2.0:1775766034.443297:0:8:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766034.443307:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766034.443320:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114422656:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766034.443329:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1880 x1862022114423040/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766034.443346:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766034.443350:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775766034.443353:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000800:00000100:3.0:1775766034.443360:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:2.0:1775766034.443366:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3411
00000100:00080000:2.0:1775766034.443374:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766034.443380:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775766034.443384:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766034/real 1775766034]  req@ffff9f14852d1c00 x1862022114422656/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766059 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:2.0:1775766034.443388:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766034.443412:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0e00 x1862022114423168/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775766034.443423:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1c00 x1862022114422656/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766059 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766034.443428:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766034.443442:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775766034.443448:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766034.443455:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766034.443459:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3411
00000100:00080000:3.0:1775766034.443466:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00080000:2.0:1775766034.443467:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:3.0:1775766034.443474:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114422656:192.168.202.110@tcp:38:kworker.0
00000100:00080000:2.0:1775766034.443477:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766034.443490:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766034.443493:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114422784:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766034.443506:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3b80 x1862022114423296/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766034.443519:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000800:00000100:3.0:1775766034.443523:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:2.0:1775766034.443529:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775766034.443534:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766034.443542:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3411
00000100:00100000:3.0:1775766034.443546:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766034/real 1775766034]  req@ffff9f14852d2300 x1862022114422784/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766059 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766034.443549:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766034.443561:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766034.443570:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766034.443574:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2300 x1862022114422784/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766059 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766034.443617:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0000 x1862022114423424/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766034.443655:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766034.443667:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766034.443677:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114422784:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766034.443690:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114422912:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775766034.443731:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766034.443746:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766034/real 1775766034]  req@ffff9f14852d3100 x1862022114422912/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766059 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766034.443759:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3100 x1862022114422912/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766059 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766034.443776:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766034.443779:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766034.443783:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114422912:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775766034.443803:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114423040:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775766034.443822:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766034.443832:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766034/real 1775766034]  req@ffff9f14852d1880 x1862022114423040/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766059 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766034.443848:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1880 x1862022114423040/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766059 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766034.443862:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766034.443867:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766034.443872:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114423040:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775766034.443881:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114423168:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775766034.443914:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766034.443933:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766034/real 1775766034]  req@ffff9f14852d0e00 x1862022114423168/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766059 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766034.443954:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0e00 x1862022114423168/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766059 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766034.443990:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766034.443994:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766034.443998:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114423168:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766034.444006:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114423296:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775766034.444034:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766034.444047:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766034/real 1775766034]  req@ffff9f14852d3b80 x1862022114423296/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766059 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766034.444061:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3b80 x1862022114423296/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766059 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766034.444071:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766034.444075:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766034.444079:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114423296:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766034.444085:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114423424:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775766034.444114:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766034.444147:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766034/real 1775766034]  req@ffff9f14852d0000 x1862022114423424/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766059 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766034.444167:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0000 x1862022114423424/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766059 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766034.444206:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766034.444213:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766034.444218:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114423424:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775766034.698817:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775766035.466687:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:3.0:1775766036.490713:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766036.746754:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775766037.514641:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775766037.770772:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775766038.538728:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775766038.794760:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1775766039.562674:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:3.0:1775766039.562676:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:2.0:1775766039.562683:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1775766039.562687:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766039.562696:0:8:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3417
00000100:00080000:2.0:1775766039.562707:0:8:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766039.562715:0:8:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775766039.562755:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0380 x1862022114423552/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766039.562787:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766039.562799:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775766039.562801:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766039.562806:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3417
00000100:00080000:2.0:1775766039.562812:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766039.562817:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766039.562826:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766039.562829:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114423552:192.168.202.110@tcp:250:kworker.0
00000100:00100000:2.0:1775766039.562859:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3800 x1862022114423680/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766039.562877:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766039.562880:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775766039.562882:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766039.562886:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3417
00000100:00080000:2.0:1775766039.562899:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766039.562902:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766039.562907:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000800:00000100:3.0:1775766039.562911:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766039.562922:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2d80 x1862022114423808/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766039.562931:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766039.562934:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775766039.562935:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766039.562939:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3417
00000100:00080000:2.0:1775766039.562941:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766039.562944:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766039.562949:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766039.562954:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766039/real 1775766039]  req@ffff9f14852d0380 x1862022114423552/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766064 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766039.562961:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0700 x1862022114423936/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766039.562982:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766039.562985:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1775766039.562988:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:3.0:1775766039.562989:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0380 x1862022114423552/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766064 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766039.562998:0:8:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3417
00000100:00080000:2.0:1775766039.563005:0:8:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766039.563012:0:8:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00080000:3.0:1775766039.563013:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000008:00100000:2.0:1775766039.563016:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00080000:3.0:1775766039.563020:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766039.563028:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0a80 x1862022114424064/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775766039.563028:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114423552:192.168.202.110@tcp:250:kworker.0
00000100:00080000:2.0:1775766039.563046:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766039.563050:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775766039.563053:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:3.0:1775766039.563055:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114423680:192.168.202.110@tcp:38:kworker.0
00000100:00080000:2.0:1775766039.563062:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3417
00000100:00080000:2.0:1775766039.563068:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766039.563075:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000800:00000100:3.0:1775766039.563080:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000008:00100000:2.0:1775766039.563082:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766039.563112:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766039/real 1775766039]  req@ffff9f14852d3800 x1862022114423680/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766064 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766039.563123:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1500 x1862022114424192/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775766039.563132:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3800 x1862022114423680/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766064 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766039.563147:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766039.563150:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766039.563153:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775766039.563159:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766039.563163:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00080000:2.0:1775766039.563169:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3417
00000100:00080000:2.0:1775766039.563175:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766039.563197:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766039.563202:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766039.563209:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114423680:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766039.563219:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2680 x1862022114424320/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775766039.563231:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114423808:192.168.202.110@tcp:38:kworker.0
00000100:00080000:2.0:1775766039.563267:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766039.563271:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775766039.563274:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000800:00000100:3.0:1775766039.563280:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:2.0:1775766039.563283:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3417
00000100:00080000:2.0:1775766039.563290:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:3.0:1775766039.563292:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766039/real 1775766039]  req@ffff9f14852d2d80 x1862022114423808/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766064 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766039.563306:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2d80 x1862022114423808/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766064 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766039.563307:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766039.563317:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00080000:3.0:1775766039.563317:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766039.563324:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766039.563326:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114423808:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766039.563331:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114423936:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775766039.563335:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2a00 x1862022114424448/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:3.0:1775766039.563352:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766039.563360:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766039/real 1775766039]  req@ffff9f14852d0700 x1862022114423936/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766064 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766039.563369:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0700 x1862022114423936/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766064 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766039.563376:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766039.563378:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766039.563380:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114423936:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775766039.563384:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114424064:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775766039.563394:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766039.563399:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766039/real 1775766039]  req@ffff9f14852d0a80 x1862022114424064/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766064 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766039.563408:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0a80 x1862022114424064/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766064 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766039.563413:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766039.563415:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766039.563417:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114424064:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775766039.563431:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114424192:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775766039.563441:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766039.563447:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766039/real 1775766039]  req@ffff9f14852d1500 x1862022114424192/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766064 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766039.563455:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1500 x1862022114424192/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766064 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766039.563461:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766039.563463:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766039.563464:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114424192:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766039.563468:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114424320:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775766039.563478:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766039.563483:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766039/real 1775766039]  req@ffff9f14852d2680 x1862022114424320/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766064 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766039.563491:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2680 x1862022114424320/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766064 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766039.563501:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766039.563503:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766039.563505:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114424320:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766039.563509:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114424448:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775766039.563518:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766039.563523:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766039/real 1775766039]  req@ffff9f14852d2a00 x1862022114424448/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766064 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766039.563530:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2a00 x1862022114424448/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766064 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766039.563535:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766039.563537:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766039.563540:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114424448:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775766039.818731:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775766040.586658:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766040.842814:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775766041.610631:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766041.866685:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775766042.634634:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:3.0:1775766043.658732:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766043.914723:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775766044.682706:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:1.0:1775766044.682742:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766044.682753:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1775766044.682758:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766044.682768:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3422
00000100:00080000:1.0:1775766044.682783:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766044.682790:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775766044.682851:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff100 x1862022114424576/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766044.682882:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766044.682885:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775766044.682886:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766044.682890:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3422
00000100:00080000:1.0:1775766044.682892:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766044.682896:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766044.682912:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775766044.682928:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fea00 x1862022114424704/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775766044.682929:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114424576:192.168.202.110@tcp:250:kworker.0
00000100:00080000:1.0:1775766044.682954:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766044.682956:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775766044.682958:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766044.682962:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3422
00000100:00080000:1.0:1775766044.682965:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766044.682968:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766044.682971:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775766044.682979:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fc380 x1862022114424832/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766044.682986:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766044.682989:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775766044.682990:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766044.682993:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3422
00000100:00080000:1.0:1775766044.682996:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766044.682999:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766044.683002:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775766044.683010:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fe300 x1862022114424960/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:3.0:1775766044.683019:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:1.0:1775766044.683030:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766044.683034:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1775766044.683036:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766044.683042:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3422
00000100:00080000:1.0:1775766044.683048:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766044.683052:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766044.683056:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775766044.683071:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fd880 x1862022114425088/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775766044.683075:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766044/real 1775766044]  req@ffff9f14ab4ff100 x1862022114424576/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766069 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766044.683121:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766044.683132:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775766044.683135:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766044.683142:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3422
00000100:00080000:1.0:1775766044.683145:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:3.0:1775766044.683149:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff100 x1862022114424576/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766069 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766044.683150:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766044.683161:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00080000:3.0:1775766044.683175:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00100000:1.0:1775766044.683180:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fce00 x1862022114425216/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766044.683184:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:1.0:1775766044.683206:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766044.683213:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775766044.683215:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:3.0:1775766044.683218:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114424576:192.168.202.110@tcp:250:kworker.0
00000100:00080000:1.0:1775766044.683221:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3422
00000100:00080000:1.0:1775766044.683227:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766044.683234:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766044.683238:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775766044.683245:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fca80 x1862022114425344/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766044.683250:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766044.683253:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775766044.683255:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766044.683258:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3422
00000100:00080000:1.0:1775766044.683260:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:3.0:1775766044.683266:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114424704:192.168.202.110@tcp:38:kworker.0
00000100:00080000:1.0:1775766044.683267:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766044.683279:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775766044.683290:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fed80 x1862022114425472/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:3.0:1775766044.683305:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766044.683322:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766044/real 1775766044]  req@ffff9f14ab4fea00 x1862022114424704/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766069 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766044.683332:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fea00 x1862022114424704/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766069 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766044.683339:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766044.683342:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766044.683344:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114424704:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766044.683350:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114424832:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775766044.683362:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766044.683368:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766044/real 1775766044]  req@ffff9f14ab4fc380 x1862022114424832/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766069 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766044.683376:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fc380 x1862022114424832/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766069 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766044.683381:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766044.683383:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766044.683385:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114424832:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766044.683391:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114424960:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775766044.683402:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766044.683408:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766044/real 1775766044]  req@ffff9f14ab4fe300 x1862022114424960/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766069 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766044.683416:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fe300 x1862022114424960/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766069 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766044.683421:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766044.683423:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766044.683425:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114424960:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775766044.683429:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114425088:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775766044.683438:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766044.683444:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766044/real 1775766044]  req@ffff9f14ab4fd880 x1862022114425088/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766069 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766044.683452:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fd880 x1862022114425088/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766069 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766044.683457:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766044.683459:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766044.683479:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114425088:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775766044.683484:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114425216:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775766044.683497:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766044.683504:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766044/real 1775766044]  req@ffff9f14ab4fce00 x1862022114425216/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766069 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766044.683513:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fce00 x1862022114425216/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766069 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766044.683519:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766044.683521:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766044.683523:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114425216:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766044.683527:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114425344:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775766044.683620:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766044.683629:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766044/real 1775766044]  req@ffff9f14ab4fca80 x1862022114425344/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766069 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766044.683655:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fca80 x1862022114425344/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766069 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766044.683664:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766044.683668:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766044.683672:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114425344:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766044.683698:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114425472:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775766044.683742:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766044.683756:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766044/real 1775766044]  req@ffff9f14ab4fed80 x1862022114425472/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766069 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766044.683767:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fed80 x1862022114425472/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766069 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766044.683796:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766044.683799:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766044.683801:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114425472:192.168.202.110@tcp:8:kworker.0
00010000:00010000:3.0:1775766044.938716:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775766045.706689:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775766045.962732:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775766046.730667:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766046.986693:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775766047.754628:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766048.010699:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775766048.778634:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766049.034708:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1775766049.802693:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766049.802701:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775766049.802705:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766049.802714:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3427
00000100:00080000:3.0:1775766049.802723:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766049.802731:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775766049.802769:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745880 x1862022114425600/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766049.802795:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766049.802798:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775766049.802801:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766049.802807:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3427
00000100:00080000:3.0:1775766049.802811:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766049.802814:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766049.802823:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766049.802835:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744700 x1862022114425728/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766049.802848:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766049.802861:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775766049.802864:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766049.802869:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3427
00000100:00080000:3.0:1775766049.802872:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766049.802876:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766049.802880:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766049.802893:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745500 x1862022114425856/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766049.802901:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766049.802903:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775766049.802904:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766049.802907:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3427
00000100:00080000:3.0:1775766049.802909:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766049.802913:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766049.802916:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766049.802923:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744000 x1862022114425984/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766049.802928:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766049.802931:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775766049.802933:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766049.802937:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3427
00000100:00080000:3.0:1775766049.802940:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766049.802943:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766049.802946:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766049.802955:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747800 x1862022114426112/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766049.803087:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766049.803090:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775766049.803092:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766049.803097:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3427
00000100:00080000:3.0:1775766049.803102:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766049.803107:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766049.803215:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766049.803233:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745c00 x1862022114426240/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766049.803259:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766049.803262:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775766049.803265:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766049.803334:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3427
00000100:00080000:3.0:1775766049.803340:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766049.803350:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766049.803357:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766049.803371:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745180 x1862022114426368/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766049.803396:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766049.803404:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775766049.803408:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766049.803423:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3427
00000100:00080000:3.0:1775766049.803434:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766049.803445:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766049.803457:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766049.803478:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745f80 x1862022114426496/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00010000:00010000:2.0:1775766051.082754:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00010000:00010000:3.0:1775766052.106751:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775766052.874727:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775766053.130838:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775766053.898709:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766054.154781:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1775766054.922729:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766054.922745:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775766054.922752:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766054.922795:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3432
00000100:00080000:3.0:1775766054.922814:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766054.922826:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775766054.922908:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747100 x1862022114426624/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766054.922943:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766054.922948:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775766054.922952:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766054.922959:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3432
00000100:00080000:3.0:1775766054.922964:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766054.922971:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766054.922991:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766054.923009:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745f80 x1862022114426752/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766054.923021:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766054.923026:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775766054.923028:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766054.923033:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3432
00000100:00080000:3.0:1775766054.923038:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766054.923042:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766054.923048:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766054.923060:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745180 x1862022114426880/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766054.923075:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766054.923079:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775766054.923081:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766054.923087:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3432
00000100:00080000:3.0:1775766054.923092:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766054.923097:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766054.923103:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766054.923121:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745c00 x1862022114427008/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766054.923148:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766054.923152:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775766054.923155:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766054.923160:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3432
00000100:00080000:3.0:1775766054.923164:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766054.923171:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766054.923181:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766054.923197:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747800 x1862022114427136/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766054.923215:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766054.923220:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775766054.923223:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766054.923228:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3432
00000100:00080000:3.0:1775766054.923234:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766054.923242:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766054.923249:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766054.923266:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744000 x1862022114427264/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766054.923288:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766054.923296:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775766054.923300:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766054.923311:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3432
00000100:00080000:3.0:1775766054.923318:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766054.923328:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766054.923349:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766054.923379:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745500 x1862022114427392/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766054.923428:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766054.923458:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775766054.923479:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766054.923487:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3432
00000100:00080000:3.0:1775766054.923493:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766054.923498:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766054.923505:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766054.923525:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744700 x1862022114427520/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00010000:00010000:2.0:1775766055.178748:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00010000:00010000:2.0:1775766056.202817:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00010000:00010000:2.0:1775766058.250863:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775766059.018779:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775766059.274681:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1775766060.042663:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766060.042673:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775766060.042678:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766060.042687:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3437
00000100:00080000:3.0:1775766060.042699:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766060.042706:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775766060.042748:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745880 x1862022114427648/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766060.042776:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766060.042780:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775766060.042781:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766060.042785:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3437
00000100:00080000:3.0:1775766060.042788:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766060.042792:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766060.042802:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766060.042812:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746680 x1862022114427776/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766060.042826:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766060.042841:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775766060.042844:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766060.042848:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3437
00000100:00080000:3.0:1775766060.042850:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766060.042853:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766060.042856:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766060.042869:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747b80 x1862022114427904/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766060.042877:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766060.042880:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775766060.042881:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766060.042886:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3437
00000100:00080000:3.0:1775766060.042890:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766060.042895:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766060.042900:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766060.042907:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744e00 x1862022114428032/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766060.042914:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766060.042917:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775766060.042918:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766060.042921:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3437
00000100:00080000:3.0:1775766060.042924:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766060.042939:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766060.042945:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766060.042965:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744380 x1862022114428160/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766060.042974:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766060.042977:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775766060.042979:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766060.042984:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3437
00000100:00080000:3.0:1775766060.042988:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766060.042993:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766060.043003:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766060.043025:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746a00 x1862022114428288/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766060.043114:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766060.043129:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775766060.043133:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766060.043142:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3437
00000100:00080000:3.0:1775766060.043147:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766060.043154:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766060.043163:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766060.043183:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746300 x1862022114428416/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766060.043193:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766060.043204:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775766060.043206:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766060.043211:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3437
00000100:00080000:3.0:1775766060.043218:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766060.043222:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766060.043226:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766060.043243:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747480 x1862022114428544/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00010000:00010000:3.0:1775766060.298722:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00010000:00010000:2.0:1775766061.322731:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00010000:00010000:2.0:1775766062.346684:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775766063.114703:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766063.370862:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1775766065.162744:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766065.162751:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1775766065.162756:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766065.162766:0:8:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3442
00000100:00080000:2.0:1775766065.162785:0:8:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766065.162792:0:8:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775766065.162831:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1180 x1862022114428672/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766065.162871:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766065.162874:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775766065.162875:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766065.162879:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3442
00000100:00080000:2.0:1775766065.162882:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766065.162885:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766065.162893:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766065.162905:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1f80 x1862022114428800/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766065.162921:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766065.162924:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775766065.162925:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766065.162928:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3442
00000100:00080000:2.0:1775766065.162931:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766065.162933:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766065.162936:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766065.162946:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2a00 x1862022114428928/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766065.162958:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766065.162960:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775766065.162961:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766065.162964:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3442
00000100:00080000:2.0:1775766065.162966:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766065.162968:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766065.162971:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775766065.162978:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2680 x1862022114429056/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766065.162990:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766065.162993:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1775766065.162994:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766065.162998:0:8:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3442
00000100:00080000:2.0:1775766065.163001:0:8:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766065.163007:0:8:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766065.163011:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775766065.163021:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1500 x1862022114429184/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766065.163029:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766065.163037:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775766065.163039:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766065.163043:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3442
00000100:00080000:2.0:1775766065.163047:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766065.163051:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766065.163055:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766065.163069:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0a80 x1862022114429312/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766065.163077:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766065.163081:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775766065.163085:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766065.163090:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3442
00000100:00080000:2.0:1775766065.163093:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766065.163096:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766065.163100:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766065.163110:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0700 x1862022114429440/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766065.163118:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766065.163121:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775766065.163123:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766065.163131:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3442
00000100:00080000:2.0:1775766065.163135:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766065.163143:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766065.163147:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775766065.163157:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2d80 x1862022114429568/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00010000:00010000:2.0:1775766065.418721:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00010000:00010000:3.0:1775766066.442718:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00010000:00010000:3.0:1775766067.466743:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000400:00000100:1.0:1775766068.234915:0:2393:0:(lib-socket.c:454:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.202.110:988
00000400:00000100:1.0:1775766068.234990:0:2393:0:(acceptor.c:111:lnet_connect_console_error()) Connection to 192.168.202.110@tcp at host 192.168.202.110 was unreachable: the network or that node may be down, or Lustre may be misconfigured.
00000800:00000100:1.0:1775766068.235013:0:2393:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:1.0:1775766068.235066:0:2393:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:1.0:1775766068.235082:0:2393:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:1.0:1775766068.235096:0:2393:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:1.0:1775766068.235114:0:2393:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:1.0:1775766068.235134:0:2393:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:1.0:1775766068.235153:0:2393:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:1.0:1775766068.235178:0:2393:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00010000:00010000:2.0:1775766068.490878:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00010000:00010000:2.0:1775766069.514803:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1775766070.282799:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766070.282806:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1775766070.282810:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766070.282818:0:8:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3447
00000100:00080000:1.0:1775766070.282829:0:8:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766070.282835:0:8:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775766070.282867:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdc00 x1862022114429696/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766070.282896:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766070.282901:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775766070.282904:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766070.282909:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3447
00000100:00080000:1.0:1775766070.282913:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766070.282919:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766070.282931:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775766070.282958:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdf80 x1862022114429824/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766070.282981:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766070.282986:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775766070.282989:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766070.282994:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3447
00000100:00080000:1.0:1775766070.283016:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766070.283023:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766070.283035:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775766070.283066:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ffb80 x1862022114429952/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766070.283086:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766070.283094:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775766070.283098:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766070.283111:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3447
00000100:00080000:1.0:1775766070.283130:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766070.283135:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766070.283144:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775766070.283167:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff800 x1862022114430080/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766070.283191:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766070.283196:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1775766070.283198:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766070.283204:0:8:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3447
00000100:00080000:1.0:1775766070.283209:0:8:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766070.283214:0:8:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766070.283222:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775766070.283237:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdc00 x1862022114430208/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766070.283258:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766070.283263:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775766070.283265:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766070.283271:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3447
00000100:00080000:1.0:1775766070.283278:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766070.283288:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766070.283314:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775766070.283334:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fed80 x1862022114430336/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766070.283364:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766070.283370:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775766070.283375:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766070.283387:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3447
00000100:00080000:1.0:1775766070.283394:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766070.283404:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766070.283415:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775766070.283435:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fca80 x1862022114430464/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766070.283450:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766070.283455:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775766070.283459:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766070.283471:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3447
00000100:00080000:1.0:1775766070.283477:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766070.283487:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766070.283499:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775766070.283519:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fce00 x1862022114430592/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775766070.283846:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766070/real 1775766070]  req@ffff9f14ab4fed80 x1862022114430336/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766095 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766070.283862:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fed80 x1862022114430336/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766095 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766070.283872:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766070.283876:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766070.283879:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114430336:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766070.283886:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114430464:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766070.283931:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766070.283945:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766070/real 1775766070]  req@ffff9f14ab4fca80 x1862022114430464/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766095 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766070.283978:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fca80 x1862022114430464/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766095 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766070.283987:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766070.283990:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766070.283994:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114430464:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766070.284002:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114430592:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766070.284027:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766070.284040:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766070/real 1775766070]  req@ffff9f14ab4fce00 x1862022114430592/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766095 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766070.284055:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fce00 x1862022114430592/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766095 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766070.284075:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766070.284079:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766070.284083:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114430592:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775766070.538729:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766071.306663:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775766071.562707:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766072.330647:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766072.586722:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775766073.354669:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775766073.610718:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775766074.378620:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775766074.634792:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1775766075.402701:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:3.0:1775766075.402701:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:2.0:1775766075.402714:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1775766075.402721:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766075.402735:0:8:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3452
00000100:00080000:2.0:1775766075.402750:0:8:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766075.402762:0:8:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775766075.402816:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3800 x1862022114430720/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766075.402860:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766075.402866:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775766075.402870:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766075.402877:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3452
00000100:00080000:2.0:1775766075.402884:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766075.402892:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766075.402907:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775766075.402928:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114430720:192.168.202.110@tcp:250:kworker.0
00000100:00100000:2.0:1775766075.402941:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0380 x1862022114430848/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766075.402956:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766075.402961:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775766075.402964:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766075.402970:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3452
00000100:00080000:2.0:1775766075.402975:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766075.402980:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766075.402986:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766075.402998:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0000 x1862022114430976/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766075.403011:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000800:00000100:0.0:1775766075.403012:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:2.0:1775766075.403020:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775766075.403023:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766075.403029:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3452
00000100:00080000:2.0:1775766075.403034:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766075.403038:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766075.403044:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775766075.403057:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3b80 x1862022114431104/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766075.403068:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1775766075.403073:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766075/real 1775766075]  req@ffff9f14852d3800 x1862022114430720/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766100 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766075.403074:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1775766075.403079:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766075.403090:0:8:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3452
00000100:00080000:2.0:1775766075.403098:0:8:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766075.403107:0:8:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766075.403114:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775766075.403121:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3800 x1862022114430720/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766100 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766075.403129:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0e00 x1862022114431232/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775766075.403150:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766075.403153:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766075.403161:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775766075.403166:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:2.0:1775766075.403168:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775766075.403181:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114430720:192.168.202.110@tcp:250:kworker.0
00000100:00080000:2.0:1775766075.403181:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3452
00000100:00080000:2.0:1775766075.403189:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766075.403198:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766075.403203:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766075.403214:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1880 x1862022114431360/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775766075.403224:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114430848:192.168.202.110@tcp:38:kworker.0
00000100:00080000:2.0:1775766075.403231:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766075.403239:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775766075.403242:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000800:00000100:0.0:1775766075.403301:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:2.0:1775766075.403308:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3452
00000100:00080000:2.0:1775766075.403318:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1775766075.403324:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766075/real 1775766075]  req@ffff9f14852d0380 x1862022114430848/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766100 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766075.403331:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775766075.403354:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0380 x1862022114430848/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766100 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:2.0:1775766075.403361:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00080000:0.0:1775766075.403378:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766075.403385:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766075.403390:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3100 x1862022114431488/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775766075.403396:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114430848:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766075.403416:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114430976:192.168.202.110@tcp:38:kworker.0
00000100:00080000:2.0:1775766075.403431:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766075.403435:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775766075.403439:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000800:00000100:0.0:1775766075.403446:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:2.0:1775766075.403453:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3452
00000100:00080000:2.0:1775766075.403460:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1775766075.403468:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766075/real 1775766075]  req@ffff9f14852d0000 x1862022114430976/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766100 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766075.403469:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766075.403481:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775766075.403498:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0000 x1862022114430976/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766100 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766075.403506:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2300 x1862022114431616/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775766075.403525:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766075.403537:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766075.403544:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114430976:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766075.403588:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114431104:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766075.403614:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766075.403630:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766075/real 1775766075]  req@ffff9f14852d3b80 x1862022114431104/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766100 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766075.403644:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3b80 x1862022114431104/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766100 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766075.403653:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766075.403657:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766075.403660:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114431104:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775766075.403666:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114431232:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766075.403692:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766075.403708:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766075/real 1775766075]  req@ffff9f14852d0e00 x1862022114431232/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766100 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766075.403729:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0e00 x1862022114431232/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766100 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766075.403743:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766075.403749:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766075.403755:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114431232:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775766075.403785:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114431360:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766075.403813:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766075.403826:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766075/real 1775766075]  req@ffff9f14852d1880 x1862022114431360/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766100 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766075.403839:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1880 x1862022114431360/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766100 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766075.403848:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766075.403852:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766075.403855:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114431360:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766075.403861:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114431488:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766075.403877:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766075.403886:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766075/real 1775766075]  req@ffff9f14852d3100 x1862022114431488/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766100 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766075.403906:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3100 x1862022114431488/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766100 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766075.403914:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766075.403916:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766075.403920:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114431488:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766075.403925:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114431616:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766075.403939:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766075.403947:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766075/real 1775766075]  req@ffff9f14852d2300 x1862022114431616/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766100 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766075.403958:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2300 x1862022114431616/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766100 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766075.403966:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766075.403969:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766075.403973:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114431616:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775766075.658815:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775766076.426673:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766076.682677:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766077.450690:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766077.706779:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766078.474691:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775766078.730710:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766079.498641:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766079.754785:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766080.522647:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775766080.522666:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766080.522674:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775766080.522679:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766080.522688:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3458
00000100:00080000:3.0:1775766080.522699:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766080.522706:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775766080.522741:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744a80 x1862022114431744/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766080.522763:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766080.522766:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775766080.522768:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766080.522771:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3458
00000100:00080000:3.0:1775766080.522773:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766080.522776:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766080.522783:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766080.522794:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747480 x1862022114431872/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766080.522803:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1775766080.522806:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114431744:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775766080.522807:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775766080.522809:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766080.522817:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3458
00000100:00080000:3.0:1775766080.522820:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766080.522822:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766080.522825:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766080.522831:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746300 x1862022114432000/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766080.522837:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766080.522840:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775766080.522841:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766080.522844:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3458
00000100:00080000:3.0:1775766080.522846:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766080.522848:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766080.522851:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766080.522858:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746a00 x1862022114432128/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766080.522871:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766080.522873:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775766080.522874:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766080.522877:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3458
00000100:00080000:3.0:1775766080.522879:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766080.522881:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766080.522885:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:0.0:1775766080.522890:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766080.522896:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744380 x1862022114432256/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766080.522905:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766080.522907:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775766080.522909:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766080.522912:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3458
00000100:00080000:3.0:1775766080.522914:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766080.522917:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766080.522920:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766080.522927:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744e00 x1862022114432384/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766080.522934:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766080.522936:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775766080.522938:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775766080.522942:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766080/real 1775766080]  req@ffff9f1485744a80 x1862022114431744/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766105 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766080.522943:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3458
00000100:00080000:3.0:1775766080.522948:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766080.522953:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766080.522961:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766080.522973:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747b80 x1862022114432512/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766080.522984:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1775766080.522988:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744a80 x1862022114431744/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766105 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766080.522991:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775766080.522995:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766080.523006:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3458
00000100:00080000:0.0:1775766080.523008:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766080.523012:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766080.523016:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:3.0:1775766080.523022:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775766080.523024:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114431744:192.168.202.110@tcp:250:kworker.0
00000008:00100000:3.0:1775766080.523031:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766080.523044:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746680 x1862022114432640/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775766080.523052:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114431872:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766080.523079:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766080.523091:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766080/real 1775766080]  req@ffff9f1485747480 x1862022114431872/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766105 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766080.523104:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747480 x1862022114431872/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766105 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766080.523112:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766080.523116:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766080.523129:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114431872:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766080.523135:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114432000:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766080.523153:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766080.523163:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766080/real 1775766080]  req@ffff9f1485746300 x1862022114432000/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766105 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766080.523176:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746300 x1862022114432000/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766105 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766080.523190:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766080.523194:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766080.523215:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114432000:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766080.523222:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114432128:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766080.523241:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766080.523252:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766080/real 1775766080]  req@ffff9f1485746a00 x1862022114432128/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766105 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766080.523264:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746a00 x1862022114432128/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766105 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766080.523273:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766080.523277:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766080.523288:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114432128:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775766080.523295:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114432256:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766080.523315:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766080.523326:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766080/real 1775766080]  req@ffff9f1485744380 x1862022114432256/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766105 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766080.523338:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744380 x1862022114432256/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766105 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766080.523346:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766080.523349:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766080.523353:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114432256:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775766080.523360:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114432384:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766080.523378:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766080.523389:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766080/real 1775766080]  req@ffff9f1485744e00 x1862022114432384/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766105 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766080.523400:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744e00 x1862022114432384/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766105 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766080.523407:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766080.523410:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766080.523413:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114432384:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766080.523418:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114432512:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766080.523434:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766080.523451:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766080/real 1775766080]  req@ffff9f1485747b80 x1862022114432512/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766105 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766080.523462:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747b80 x1862022114432512/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766105 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766080.523470:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766080.523473:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766080.523476:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114432512:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766080.523495:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114432640:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766080.523514:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766080.523523:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766080/real 1775766080]  req@ffff9f1485746680 x1862022114432640/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766105 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766080.523535:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746680 x1862022114432640/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766105 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766080.523542:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766080.523545:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766080.523549:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114432640:192.168.202.110@tcp:8:kworker.0
00010000:00010000:3.0:1775766080.778726:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766081.546657:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775766081.802745:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766082.570613:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766082.826757:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775766083.594753:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766083.850782:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775766084.618646:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766084.874769:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1775766085.642666:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766085.642675:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775766085.642680:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775766085.642688:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775766085.642690:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3463
00000100:00080000:3.0:1775766085.642703:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766085.642709:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775766085.642749:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745880 x1862022114432768/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766085.642769:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766085.642772:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775766085.642774:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766085.642780:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3463
00000100:00080000:3.0:1775766085.642785:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766085.642790:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766085.642801:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775766085.642802:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114432768:192.168.202.110@tcp:250:kworker.0
00000100:00100000:3.0:1775766085.642822:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744700 x1862022114432896/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766085.642835:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766085.642839:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775766085.642842:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766085.642848:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3463
00000100:00080000:3.0:1775766085.642855:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766085.642860:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000800:00000100:0.0:1775766085.642864:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000008:00100000:3.0:1775766085.642869:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766085.642884:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745500 x1862022114433024/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766085.642892:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766085.642895:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775766085.642897:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775766085.642904:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766085/real 1775766085]  req@ffff9f1485745880 x1862022114432768/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766110 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766085.642904:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3463
00000100:00080000:3.0:1775766085.642911:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766085.642920:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766085.642926:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775766085.642928:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745880 x1862022114432768/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766110 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766085.642937:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766085.642942:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766085.642945:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114432768:192.168.202.110@tcp:250:kworker.0
00000100:00100000:0.0:1775766085.642963:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114432896:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766085.642980:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766085.642991:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766085/real 1775766085]  req@ffff9f1485744700 x1862022114432896/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766110 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766085.643007:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744000 x1862022114433152/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775766085.643008:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744700 x1862022114432896/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766110 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766085.643038:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766085.643041:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766085.643043:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00080000:3.0:1775766085.643047:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00100000:0.0:1775766085.643050:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114432896:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775766085.643051:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766085.643059:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3463
00000100:00100000:0.0:1775766085.643061:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114433024:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775766085.643064:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766085.643069:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766085.643074:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:0.0:1775766085.643077:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766085.643088:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766085/real 1775766085]  req@ffff9f1485745500 x1862022114433024/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766110 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766085.643095:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745500 x1862022114433024/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766110 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766085.643100:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766085.643102:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766085.643104:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114433024:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766085.643114:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114433152:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766085.643127:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766085.643132:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766085/real 1775766085]  req@ffff9f1485744000 x1862022114433152/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766110 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766085.643139:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744000 x1862022114433152/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766110 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766085.643168:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766085.643170:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766085.643173:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114433152:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775766085.643234:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747800 x1862022114433280/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766085.643253:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766085.643258:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775766085.643260:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766085.643264:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3463
00000100:00080000:3.0:1775766085.643270:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1775766085.643284:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114433280:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766085.643305:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766085.643317:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766085/real 1775766085]  req@ffff9f1485747800 x1862022114433280/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766110 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766085.643335:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747800 x1862022114433280/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766110 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766085.643346:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00080000:0.0:1775766085.643347:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766085.643353:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766085.643356:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114433280:192.168.202.110@tcp:8:kworker.0
00000008:00100000:3.0:1775766085.643358:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766085.643375:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745c00 x1862022114433408/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766085.643392:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766085.643407:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775766085.643409:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775766085.643414:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114433408:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775766085.643415:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3463
00000100:00080000:3.0:1775766085.643423:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766085.643428:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000800:00000100:0.0:1775766085.643434:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000008:00100000:3.0:1775766085.643434:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775766085.643444:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766085/real 1775766085]  req@ffff9f1485745c00 x1862022114433408/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766110 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766085.643450:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745180 x1862022114433536/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775766085.643460:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745c00 x1862022114433408/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766110 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766085.643478:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766085.643484:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766085.643489:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114433408:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775766085.643490:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766085.643497:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775766085.643499:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775766085.643503:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114433536:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775766085.643505:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3463
00000100:00080000:3.0:1775766085.643512:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766085.643516:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000800:00000100:0.0:1775766085.643521:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000008:00100000:3.0:1775766085.643522:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775766085.643529:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766085/real 1775766085]  req@ffff9f1485745180 x1862022114433536/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766110 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766085.643546:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745f80 x1862022114433664/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775766085.643597:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745180 x1862022114433536/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766110 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766085.643609:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766085.643614:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766085.643619:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114433536:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766085.643633:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114433664:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766085.643651:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766085.643658:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766085/real 1775766085]  req@ffff9f1485745f80 x1862022114433664/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766110 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766085.643666:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745f80 x1862022114433664/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766110 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766085.643671:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766085.643674:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766085.643676:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114433664:192.168.202.110@tcp:8:kworker.0
00010000:00010000:1.0:1775766085.898707:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766086.666671:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766086.922735:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766087.690648:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775766087.946742:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766088.714662:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775766088.970754:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766089.738679:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766089.994714:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1775766090.762650:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766090.762661:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775766090.762667:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775766090.762668:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775766090.762679:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3468
00000100:00080000:3.0:1775766090.762693:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766090.762701:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775766090.762746:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747100 x1862022114433792/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766090.762771:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766090.762780:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775766090.762782:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766090.762785:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3468
00000100:00080000:3.0:1775766090.762790:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766090.762794:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766090.762804:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766090.762816:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745f80 x1862022114433920/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775766090.762827:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114433792:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775766090.762827:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766090.762833:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775766090.762836:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766090.762841:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3468
00000100:00080000:3.0:1775766090.762843:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766090.762845:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766090.762848:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766090.762855:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745180 x1862022114434048/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766090.762861:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766090.762864:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775766090.762866:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766090.762869:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3468
00000100:00080000:3.0:1775766090.762871:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766090.762874:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766090.762877:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766090.762885:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745c00 x1862022114434176/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766090.762896:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766090.762899:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775766090.762900:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766090.762903:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3468
00000100:00080000:3.0:1775766090.762906:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766090.762908:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766090.762911:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766090.762919:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747800 x1862022114434304/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1775766090.762923:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775766090.762932:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766090.762934:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775766090.762936:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766090.762940:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3468
00000100:00080000:3.0:1775766090.762943:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766090.762947:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766090.762950:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766090.762957:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744000 x1862022114434432/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766090.762962:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766090.762965:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775766090.762966:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766090.762970:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3468
00000100:00080000:3.0:1775766090.762973:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766090.762977:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775766090.762981:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766090/real 1775766090]  req@ffff9f1485747100 x1862022114433792/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766115 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:3.0:1775766090.762982:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766090.762995:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745500 x1862022114434560/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766090.763002:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766090.763005:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775766090.763006:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766090.763009:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3468
00000100:00080000:3.0:1775766090.763011:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1775766090.763015:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747100 x1862022114433792/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766115 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766090.763016:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766090.763021:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766090.763032:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744700 x1862022114434688/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775766090.763037:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766090.763045:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766090.763051:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114433792:192.168.202.110@tcp:250:kworker.0
00000100:00100000:0.0:1775766090.763078:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114433920:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766090.763160:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766090.763175:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766090/real 1775766090]  req@ffff9f1485745f80 x1862022114433920/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766115 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766090.763187:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745f80 x1862022114433920/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766115 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766090.763195:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766090.763199:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766090.763202:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114433920:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766090.763210:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114434048:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766090.763232:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766090.763242:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766090/real 1775766090]  req@ffff9f1485745180 x1862022114434048/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766115 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766090.763253:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745180 x1862022114434048/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766115 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766090.763261:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766090.763264:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766090.763267:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114434048:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766090.763274:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114434176:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766090.763295:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766090.763306:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766090/real 1775766090]  req@ffff9f1485745c00 x1862022114434176/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766115 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766090.763318:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745c00 x1862022114434176/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766115 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766090.763327:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766090.763330:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766090.763334:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114434176:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775766090.763340:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114434304:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766090.763361:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766090.763371:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766090/real 1775766090]  req@ffff9f1485747800 x1862022114434304/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766115 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766090.763390:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747800 x1862022114434304/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766115 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766090.763398:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766090.763401:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766090.763404:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114434304:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775766090.763410:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114434432:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766090.763428:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766090.763438:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766090/real 1775766090]  req@ffff9f1485744000 x1862022114434432/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766115 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766090.763450:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744000 x1862022114434432/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766115 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766090.763458:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766090.763461:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766090.763464:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114434432:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766090.763470:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114434560:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766090.763488:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766090.763498:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766090/real 1775766090]  req@ffff9f1485745500 x1862022114434560/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766115 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766090.763509:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745500 x1862022114434560/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766115 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766090.763524:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766090.763527:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766090.763531:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114434560:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766090.763536:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114434688:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766090.763555:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766090.763564:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766090/real 1775766090]  req@ffff9f1485744700 x1862022114434688/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766115 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766090.763602:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744700 x1862022114434688/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766115 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766090.763610:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766090.763613:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766090.763617:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114434688:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775766091.018776:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766091.786694:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766092.042821:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766092.810704:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775766093.066788:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775766093.834719:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766094.090835:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775766094.858750:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775766095.114821:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775766095.882774:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:2.0:1775766095.882784:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766095.882796:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1775766095.882802:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766095.882816:0:8:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3473
00000100:00080000:2.0:1775766095.882832:0:8:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766095.882843:0:8:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775766095.882922:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1c00 x1862022114434816/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766095.882965:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766095.882970:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775766095.882973:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766095.882978:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3473
00000100:00080000:2.0:1775766095.883014:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766095.883019:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766095.883034:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766095.883071:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3480 x1862022114434944/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766095.883111:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766095.883117:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775766095.883120:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775766095.883129:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114434816:192.168.202.110@tcp:250:kworker.0
00000100:00080000:2.0:1775766095.883131:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3473
00000100:00080000:2.0:1775766095.883141:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766095.883180:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766095.883189:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766095.883206:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2300 x1862022114435072/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766095.883219:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766095.883239:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775766095.883241:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766095.883245:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3473
00000100:00080000:2.0:1775766095.883250:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766095.883253:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766095.883259:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775766095.883270:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3100 x1862022114435200/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766095.883280:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000800:00000100:0.0:1775766095.883286:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:2.0:1775766095.883286:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1775766095.883292:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766095.883296:0:8:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3473
00000100:00080000:2.0:1775766095.883300:0:8:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766095.883303:0:8:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766095.883308:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775766095.883320:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1880 x1862022114435328/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766095.883328:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766095.883332:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775766095.883334:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766095.883338:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3473
00000100:00080000:2.0:1775766095.883341:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766095.883346:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775766095.883352:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766095/real 1775766095]  req@ffff9f14852d1c00 x1862022114434816/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766120 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:2.0:1775766095.883353:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766095.883392:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0e00 x1862022114435456/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775766095.883402:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1c00 x1862022114434816/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766120 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766095.883404:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766095.883415:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775766095.883420:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766095.883428:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766095.883428:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3473
00000100:00080000:2.0:1775766095.883436:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766095.883440:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:2.0:1775766095.883445:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775766095.883451:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114434816:192.168.202.110@tcp:250:kworker.0
00000008:00100000:2.0:1775766095.883452:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775766095.883467:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114434944:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766095.883473:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3b80 x1862022114435584/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766095.883489:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766095.883494:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775766095.883498:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000800:00000100:0.0:1775766095.883500:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:2.0:1775766095.883511:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3473
00000100:00080000:2.0:1775766095.883518:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1775766095.883519:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766095/real 1775766095]  req@ffff9f14852d3480 x1862022114434944/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766120 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766095.883532:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775766095.883539:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3480 x1862022114434944/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766120 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:2.0:1775766095.883583:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775766095.883601:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0000 x1862022114435712/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775766095.883625:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766095.883633:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766095.883640:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114434944:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766095.883673:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114435072:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766095.883693:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766095.883708:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766095/real 1775766095]  req@ffff9f14852d2300 x1862022114435072/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766120 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766095.883719:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2300 x1862022114435072/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766120 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766095.883726:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766095.883729:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766095.883732:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114435072:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766095.883736:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114435200:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766095.883752:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766095.883759:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766095/real 1775766095]  req@ffff9f14852d3100 x1862022114435200/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766120 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766095.883769:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3100 x1862022114435200/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766120 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766095.883776:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766095.883778:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766095.883780:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114435200:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775766095.883784:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114435328:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766095.883796:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766095.883803:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766095/real 1775766095]  req@ffff9f14852d1880 x1862022114435328/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766120 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766095.883812:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1880 x1862022114435328/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766120 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766095.883819:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766095.883821:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766095.883823:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114435328:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775766095.883837:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114435456:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766095.883853:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766095.883861:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766095/real 1775766095]  req@ffff9f14852d0e00 x1862022114435456/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766120 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766095.883878:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0e00 x1862022114435456/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766120 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766095.883885:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766095.883888:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766095.883891:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114435456:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766095.883895:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114435584:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766095.883908:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766095.883916:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766095/real 1775766095]  req@ffff9f14852d3b80 x1862022114435584/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766120 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766095.883925:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3b80 x1862022114435584/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766120 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766095.883932:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766095.883934:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766095.883936:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114435584:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766095.883941:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114435712:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766095.883954:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766095.883960:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766095/real 1775766095]  req@ffff9f14852d0000 x1862022114435712/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766120 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766095.883970:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0000 x1862022114435712/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766120 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766095.883992:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766095.883995:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766095.883998:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114435712:192.168.202.110@tcp:8:kworker.0
00010000:00010000:3.0:1775766096.138743:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775766096.906697:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766097.162884:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775766097.930725:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766098.186824:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775766098.954725:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766099.210874:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775766099.978693:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775766100.234816:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775766101.002714:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:1.0:1775766101.002787:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766101.002799:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1775766101.002804:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766101.002816:0:8:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3478
00000100:00080000:1.0:1775766101.002830:0:8:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766101.002839:0:8:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775766101.002885:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fd880 x1862022114435840/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766101.002926:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766101.002931:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775766101.002934:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766101.002939:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3478
00000100:00080000:1.0:1775766101.002943:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766101.002947:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766101.002959:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775766101.002976:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fe300 x1862022114435968/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766101.002990:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766101.002998:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775766101.003026:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775766101.003034:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114435840:192.168.202.110@tcp:250:kworker.0
00000100:00080000:1.0:1775766101.003040:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3478
00000100:00080000:1.0:1775766101.003057:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766101.003065:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766101.003076:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775766101.003104:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fc380 x1862022114436096/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766101.003128:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766101.003133:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775766101.003135:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000800:00000100:0.0:1775766101.003146:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:1.0:1775766101.003146:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3478
00000100:00080000:1.0:1775766101.003157:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766101.003166:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766101.003176:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775766101.003193:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fea00 x1862022114436224/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766101.003206:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766101.003211:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1775766101.003214:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775766101.003224:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766101/real 1775766101]  req@ffff9f14ab4fd880 x1862022114435840/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766126 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766101.003224:0:8:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3478
00000100:00080000:1.0:1775766101.003233:0:8:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766101.003243:0:8:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766101.003254:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775766101.003279:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff100 x1862022114436352/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775766101.003292:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fd880 x1862022114435840/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766126 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766101.003298:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766101.003313:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775766101.003322:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766101.003333:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775766101.003339:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3478
00000100:00080000:0.0:1775766101.003346:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:1.0:1775766101.003351:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1775766101.003355:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114435840:192.168.202.110@tcp:250:kworker.0
00000100:00080000:1.0:1775766101.003370:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775766101.003375:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114435968:192.168.202.110@tcp:38:kworker.0
00000008:00100000:1.0:1775766101.003389:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000800:00000100:0.0:1775766101.003424:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775766101.003424:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fce00 x1862022114436480/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766101.003450:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1775766101.003452:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766101/real 1775766101]  req@ffff9f14ab4fe300 x1862022114435968/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766126 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766101.003460:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775766101.003466:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766101.003480:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3478
00000100:00080000:1.0:1775766101.003497:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766101.003509:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775766101.003511:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fe300 x1862022114435968/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766126 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:1.0:1775766101.003520:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00080000:0.0:1775766101.003545:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00100000:1.0:1775766101.003550:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fca80 x1862022114436608/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766101.003610:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766101.003615:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775766101.003621:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766101.003626:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00080000:1.0:1775766101.003633:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3478
00000100:00080000:1.0:1775766101.003641:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1775766101.003643:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114435968:192.168.202.110@tcp:38:kworker.0
00000100:00080000:1.0:1775766101.003651:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766101.003666:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775766101.003684:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fed80 x1862022114436736/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775766101.003708:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114436096:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766101.003759:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766101.003779:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766101/real 1775766101]  req@ffff9f14ab4fc380 x1862022114436096/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766126 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766101.003795:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fc380 x1862022114436096/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766126 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766101.003806:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766101.003811:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766101.003815:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114436096:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766101.003823:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114436224:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766101.003845:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766101.003856:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766101/real 1775766101]  req@ffff9f14ab4fea00 x1862022114436224/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766126 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766101.003871:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fea00 x1862022114436224/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766126 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766101.003881:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766101.003884:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766101.003888:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114436224:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775766101.003894:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114436352:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766101.003913:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766101.003923:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766101/real 1775766101]  req@ffff9f14ab4ff100 x1862022114436352/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766126 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766101.003937:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff100 x1862022114436352/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766126 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766101.003947:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766101.003950:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766101.003954:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114436352:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775766101.003960:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114436480:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766101.003977:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766101.003987:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766101/real 1775766101]  req@ffff9f14ab4fce00 x1862022114436480/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766126 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766101.004032:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fce00 x1862022114436480/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766126 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766101.004043:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766101.004047:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766101.004050:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114436480:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766101.004058:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114436608:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766101.004084:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766101.004098:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766101/real 1775766101]  req@ffff9f14ab4fca80 x1862022114436608/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766126 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766101.004112:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fca80 x1862022114436608/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766126 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766101.004130:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766101.004134:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766101.004138:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114436608:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766101.004160:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114436736:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766101.004183:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766101.004200:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766101/real 1775766101]  req@ffff9f14ab4fed80 x1862022114436736/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766126 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766101.004223:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fed80 x1862022114436736/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766126 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766101.004240:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766101.004247:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766101.004254:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114436736:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775766101.258801:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775766102.026719:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775766102.282830:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775766103.050680:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775766103.306868:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775766104.074682:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766104.330765:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775766105.098706:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766105.354726:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775766106.122638:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:1.0:1775766106.122650:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766106.122657:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1775766106.122661:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766106.122671:0:8:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3483
00000100:00080000:1.0:1775766106.122683:0:8:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766106.122690:0:8:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775766106.122724:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdc00 x1862022114436864/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766106.122750:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766106.122754:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775766106.122756:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766106.122760:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3483
00000100:00080000:1.0:1775766106.122764:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766106.122767:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766106.122776:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775766106.122788:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff800 x1862022114436992/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766106.122799:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1775766106.122800:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114436864:192.168.202.110@tcp:250:kworker.0
00000100:00080000:1.0:1775766106.122808:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775766106.122811:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766106.122815:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3483
00000100:00080000:1.0:1775766106.122818:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766106.122822:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766106.122837:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775766106.122851:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ffb80 x1862022114437120/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766106.122861:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766106.122864:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775766106.122866:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000800:00000100:0.0:1775766106.122872:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:1.0:1775766106.122875:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3483
00000100:00080000:1.0:1775766106.122881:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766106.122886:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766106.122892:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775766106.122905:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdf80 x1862022114437248/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766106.122915:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1775766106.122920:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766106/real 1775766106]  req@ffff9f14ab4fdc00 x1862022114436864/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766131 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766106.122921:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1775766106.122924:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766106.122933:0:8:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3483
00000100:00080000:1.0:1775766106.122940:0:8:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766106.122944:0:8:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766106.122949:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775766106.122973:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdc00 x1862022114436864/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766131 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775766106.123001:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fed80 x1862022114437376/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775766106.123003:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766106.123015:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766106.123024:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114436864:192.168.202.110@tcp:250:kworker.0
00000100:00080000:1.0:1775766106.123024:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766106.123036:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775766106.123039:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766106.123047:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3483
00000100:00080000:1.0:1775766106.123051:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1775766106.123059:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114436992:192.168.202.110@tcp:38:kworker.0
00000100:00080000:1.0:1775766106.123061:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766106.123074:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775766106.123092:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fca80 x1862022114437504/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1775766106.123100:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:1.0:1775766106.123115:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1775766106.123118:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766106/real 1775766106]  req@ffff9f14ab4ff800 x1862022114436992/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766131 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766106.123132:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff800 x1862022114436992/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766131 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766106.123132:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775766106.123138:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766106.123150:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775766106.123151:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3483
00000100:00080000:0.0:1775766106.123160:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00080000:1.0:1775766106.123162:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1775766106.123164:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114436992:192.168.202.110@tcp:38:kworker.0
00000100:00080000:1.0:1775766106.123172:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775766106.123174:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ffb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114437120:192.168.202.110@tcp:38:kworker.0
00000008:00100000:1.0:1775766106.123184:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000800:00000100:0.0:1775766106.123211:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775766106.123218:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fce00 x1862022114437632/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775766106.123222:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766106/real 1775766106]  req@ffff9f14ab4ffb80 x1862022114437120/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766131 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766106.123247:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766106.123252:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00100000:0.0:1775766106.123253:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ffb80 x1862022114437120/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766131 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766106.123257:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766106.123266:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3483
00000100:00080000:0.0:1775766106.123272:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775766106.123273:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766106.123281:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00080000:0.0:1775766106.123282:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766106.123290:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ffb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114437120:192.168.202.110@tcp:38:kworker.0
00000008:00100000:1.0:1775766106.123291:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775766106.123301:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114437248:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775766106.123307:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff100 x1862022114437760/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1775766106.123328:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766106.123343:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766106/real 1775766106]  req@ffff9f14ab4fdf80 x1862022114437248/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766131 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766106.123354:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdf80 x1862022114437248/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766131 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766106.123360:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766106.123362:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766106.123364:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114437248:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775766106.123368:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114437376:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766106.123382:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766106.123389:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766106/real 1775766106]  req@ffff9f14ab4fed80 x1862022114437376/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766131 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766106.123396:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fed80 x1862022114437376/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766131 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766106.123401:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766106.123403:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766106.123405:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114437376:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775766106.123417:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114437504:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766106.123436:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766106.123443:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766106/real 1775766106]  req@ffff9f14ab4fca80 x1862022114437504/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766131 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766106.123450:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fca80 x1862022114437504/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766131 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766106.123460:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766106.123462:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766106.123464:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114437504:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766106.123467:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114437632:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766106.123477:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766106.123482:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766106/real 1775766106]  req@ffff9f14ab4fce00 x1862022114437632/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766131 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766106.123489:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fce00 x1862022114437632/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766131 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766106.123494:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766106.123496:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766106.123498:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114437632:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766106.123501:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114437760:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766106.123509:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766106.123514:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766106/real 1775766106]  req@ffff9f14ab4ff100 x1862022114437760/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766131 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766106.123522:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff100 x1862022114437760/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766131 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766106.123526:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766106.123528:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766106.123530:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114437760:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775766106.378694:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775766107.146628:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775766107.402722:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775766108.170704:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766108.426687:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775766109.194715:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775766109.450691:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775766110.218647:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775766110.474673:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775766111.242672:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:1.0:1775766111.242687:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766111.242695:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1775766111.242700:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766111.242711:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3488
00000100:00080000:1.0:1775766111.242722:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766111.242731:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775766111.242773:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fea00 x1862022114437888/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766111.242797:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766111.242800:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775766111.242802:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766111.242806:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3488
00000100:00080000:1.0:1775766111.242809:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766111.242828:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766111.242836:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775766111.242845:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fc380 x1862022114438016/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766111.242853:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766111.242856:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775766111.242857:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766111.242860:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3488
00000100:00080000:1.0:1775766111.242863:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1775766111.242865:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114437888:192.168.202.110@tcp:250:kworker.0
00000100:00080000:1.0:1775766111.242867:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766111.242873:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775766111.242880:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fe300 x1862022114438144/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766111.242886:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766111.242888:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775766111.242889:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766111.242891:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3488
00000100:00080000:1.0:1775766111.242893:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766111.242895:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766111.242898:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775766111.242928:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fd880 x1862022114438272/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766111.242934:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766111.242937:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1775766111.242938:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766111.242940:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3488
00000100:00080000:1.0:1775766111.242944:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000800:00000100:0.0:1775766111.242947:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:1.0:1775766111.242950:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766111.242954:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775766111.242964:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff100 x1862022114438400/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766111.242970:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766111.242972:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775766111.242973:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766111.242976:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3488
00000100:00080000:1.0:1775766111.242978:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766111.242981:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766111.242985:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775766111.242989:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766111/real 1775766111]  req@ffff9f14ab4fea00 x1862022114437888/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766136 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775766111.242994:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fce00 x1862022114438528/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766111.243011:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766111.243014:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775766111.243015:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775766111.243018:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fea00 x1862022114437888/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766136 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766111.243022:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3488
00000100:00080000:1.0:1775766111.243027:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766111.243033:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00080000:0.0:1775766111.243036:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000008:00100000:1.0:1775766111.243036:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00080000:0.0:1775766111.243053:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775766111.243054:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fca80 x1862022114438656/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775766111.243061:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114437888:192.168.202.110@tcp:250:kworker.0
00000100:00100000:0.0:1775766111.243073:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114438016:192.168.202.110@tcp:38:kworker.0
00000100:00080000:1.0:1775766111.243083:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766111.243086:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775766111.243087:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766111.243091:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3488
00000100:00080000:1.0:1775766111.243095:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000800:00000100:0.0:1775766111.243116:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:1.0:1775766111.243121:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766111.243126:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775766111.243134:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766111/real 1775766111]  req@ffff9f14ab4fc380 x1862022114438016/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766136 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775766111.243151:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fed80 x1862022114438784/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775766111.243165:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fc380 x1862022114438016/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766136 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766111.243174:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766111.243178:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766111.243183:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114438016:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766111.243204:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114438144:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766111.243221:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766111.243229:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766111/real 1775766111]  req@ffff9f14ab4fe300 x1862022114438144/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766136 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766111.243239:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fe300 x1862022114438144/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766136 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766111.243246:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766111.243249:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766111.243252:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114438144:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766111.243272:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114438272:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766111.243283:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766111.243288:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766111/real 1775766111]  req@ffff9f14ab4fd880 x1862022114438272/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766136 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766111.243296:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fd880 x1862022114438272/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766136 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766111.243304:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766111.243307:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766111.243310:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114438272:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775766111.243316:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114438400:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766111.243335:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766111.243346:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766111/real 1775766111]  req@ffff9f14ab4ff100 x1862022114438400/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766136 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766111.243359:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff100 x1862022114438400/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766136 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766111.243366:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766111.243369:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766111.243371:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114438400:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775766111.243375:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114438528:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766111.243390:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766111.243396:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766111/real 1775766111]  req@ffff9f14ab4fce00 x1862022114438528/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766136 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766111.243404:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fce00 x1862022114438528/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766136 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766111.243409:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766111.243411:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766111.243413:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114438528:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766111.243416:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114438656:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766111.243435:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766111.243440:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766111/real 1775766111]  req@ffff9f14ab4fca80 x1862022114438656/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766136 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766111.243447:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fca80 x1862022114438656/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766136 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766111.243451:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766111.243453:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766111.243455:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114438656:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766111.243459:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114438784:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766111.243468:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766111.243473:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766111/real 1775766111]  req@ffff9f14ab4fed80 x1862022114438784/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766136 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766111.243480:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fed80 x1862022114438784/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766136 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766111.243485:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766111.243486:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766111.243488:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114438784:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775766111.498710:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775766112.266672:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766112.522710:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775766113.290645:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766113.546678:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775766114.314637:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775766114.570703:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775766115.338672:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766115.594773:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775766116.362693:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:2.0:1775766116.362750:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766116.362760:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1775766116.362766:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766116.362778:0:8:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3493
00000100:00080000:2.0:1775766116.362793:0:8:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766116.362801:0:8:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775766116.362851:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0380 x1862022114438912/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766116.362945:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766116.362950:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775766116.362952:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766116.362959:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3493
00000100:00080000:2.0:1775766116.362964:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766116.362969:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766116.362982:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766116.363002:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3800 x1862022114439040/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766116.363017:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766116.363021:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775766116.363023:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766116.363030:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3493
00000100:00100000:0.0:1775766116.363037:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114438912:192.168.202.110@tcp:250:kworker.0
00000100:00080000:2.0:1775766116.363037:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766116.363045:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766116.363057:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766116.363071:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0700 x1862022114439168/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766116.363085:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766116.363089:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775766116.363092:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766116.363096:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3493
00000100:00080000:2.0:1775766116.363102:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766116.363106:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766116.363111:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775766116.363127:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2d80 x1862022114439296/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1775766116.363139:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:2.0:1775766116.363150:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766116.363155:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1775766116.363157:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766116.363172:0:8:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3493
00000100:00080000:2.0:1775766116.363177:0:8:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766116.363183:0:8:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766116.363189:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775766116.363205:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1180 x1862022114439424/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766116.363217:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1775766116.363220:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766116/real 1775766116]  req@ffff9f14852d0380 x1862022114438912/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766141 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766116.363222:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775766116.363228:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766116.363242:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3493
00000100:00080000:2.0:1775766116.363251:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766116.363258:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766116.363265:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775766116.363271:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0380 x1862022114438912/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766141 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766116.363285:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2680 x1862022114439552/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775766116.363295:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766116.363311:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766116.363323:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114438912:192.168.202.110@tcp:250:kworker.0
00000100:00080000:2.0:1775766116.363324:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766116.363339:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00100000:0.0:1775766116.363345:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114439040:192.168.202.110@tcp:38:kworker.0
00000100:00080000:2.0:1775766116.363346:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766116.363363:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3493
00000100:00080000:2.0:1775766116.363373:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766116.363384:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000800:00000100:0.0:1775766116.363390:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000008:00100000:2.0:1775766116.363401:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775766116.363412:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766116/real 1775766116]  req@ffff9f14852d3800 x1862022114439040/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766141 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766116.363430:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2a00 x1862022114439680/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766116.363446:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1775766116.363452:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3800 x1862022114439040/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766141 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766116.363453:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775766116.363465:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766116.363480:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3493
00000100:00080000:2.0:1775766116.363495:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766116.363500:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766116.363506:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00080000:0.0:1775766116.363517:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000008:00100000:2.0:1775766116.363521:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775766116.363528:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114439040:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766116.363615:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1f80 x1862022114439808/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775766116.363628:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114439168:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766116.363673:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766116.363690:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766116/real 1775766116]  req@ffff9f14852d0700 x1862022114439168/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766141 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766116.363705:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0700 x1862022114439168/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766141 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766116.363716:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766116.363720:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766116.363724:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114439168:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766116.363731:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114439296:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766116.363749:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766116.363760:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766116/real 1775766116]  req@ffff9f14852d2d80 x1862022114439296/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766141 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766116.363775:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2d80 x1862022114439296/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766141 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766116.363791:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766116.363797:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766116.363804:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114439296:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775766116.363815:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114439424:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766116.363877:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766116.363901:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766116/real 1775766116]  req@ffff9f14852d1180 x1862022114439424/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766141 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766116.363926:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1180 x1862022114439424/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766141 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766116.363942:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766116.363946:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766116.363950:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114439424:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775766116.363958:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114439552:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766116.363987:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766116.364001:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766116/real 1775766116]  req@ffff9f14852d2680 x1862022114439552/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766141 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766116.364015:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2680 x1862022114439552/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766141 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766116.364025:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766116.364028:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766116.364033:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114439552:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766116.364040:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114439680:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766116.364058:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766116.364067:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766116/real 1775766116]  req@ffff9f14852d2a00 x1862022114439680/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766141 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766116.364080:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2a00 x1862022114439680/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766141 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766116.364090:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766116.364093:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766116.364096:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114439680:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766116.364119:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114439808:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766116.364138:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766116.364148:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766116/real 1775766116]  req@ffff9f14852d1f80 x1862022114439808/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766141 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766116.364162:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1f80 x1862022114439808/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766141 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766116.364178:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766116.364181:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766116.364184:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114439808:192.168.202.110@tcp:8:kworker.0
00010000:00010000:3.0:1775766116.618919:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775766117.386682:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775766117.642702:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775766118.410625:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766118.666698:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775766119.434665:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766119.690786:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775766120.458677:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766120.714702:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775766121.482646:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:2.0:1775766121.482681:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766121.482690:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1775766121.482695:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766121.482707:0:8:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3498
00000100:00080000:2.0:1775766121.482720:0:8:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766121.482728:0:8:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775766121.482772:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0a80 x1862022114439936/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766121.482832:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766121.482839:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775766121.482851:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766121.482860:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3498
00000100:00080000:2.0:1775766121.482867:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766121.482874:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766121.482890:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775766121.482903:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114439936:192.168.202.110@tcp:250:kworker.0
00000100:00100000:2.0:1775766121.482916:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1500 x1862022114440064/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766121.482938:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766121.482944:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775766121.482949:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766121.482965:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3498
00000100:00080000:2.0:1775766121.482971:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766121.482976:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766121.482984:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000800:00000100:0.0:1775766121.482993:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766121.483004:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1f80 x1862022114440192/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766121.483017:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766121.483020:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775766121.483022:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766121.483026:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3498
00000100:00080000:2.0:1775766121.483030:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766121.483034:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766121.483040:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775766121.483047:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766121/real 1775766121]  req@ffff9f14852d0a80 x1862022114439936/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766146 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766121.483055:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2a00 x1862022114440320/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766121.483085:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766121.483089:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1775766121.483091:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775766121.483096:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0a80 x1862022114439936/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766146 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766121.483101:0:8:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3498
00000100:00080000:2.0:1775766121.483110:0:8:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766121.483119:0:8:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766121.483130:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00080000:0.0:1775766121.483131:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766121.483142:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766121.483146:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2680 x1862022114440448/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775766121.483153:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114439936:192.168.202.110@tcp:250:kworker.0
00000100:00080000:2.0:1775766121.483172:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766121.483176:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775766121.483178:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766121.483184:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3498
00000100:00080000:2.0:1775766121.483188:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1775766121.483192:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114440064:192.168.202.110@tcp:38:kworker.0
00000100:00080000:2.0:1775766121.483199:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766121.483205:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766121.483219:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1180 x1862022114440576/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766121.483231:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766121.483235:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775766121.483237:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766121.483241:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3498
00000800:00000100:0.0:1775766121.483247:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:2.0:1775766121.483250:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766121.483256:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766121.483263:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775766121.483273:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766121/real 1775766121]  req@ffff9f14852d1500 x1862022114440064/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766146 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766121.483279:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2d80 x1862022114440704/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766121.483320:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1775766121.483327:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1500 x1862022114440064/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766146 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766121.483328:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775766121.483333:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766121.483347:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3498
00000100:00080000:0.0:1775766121.483353:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766121.483354:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766121.483366:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00080000:0.0:1775766121.483368:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000008:00100000:2.0:1775766121.483379:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775766121.483384:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114440064:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766121.483396:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0700 x1862022114440832/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775766121.483398:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114440192:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766121.483436:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766121.483457:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766121/real 1775766121]  req@ffff9f14852d1f80 x1862022114440192/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766146 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766121.483482:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1f80 x1862022114440192/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766146 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766121.483510:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766121.483520:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766121.483528:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114440192:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766121.483541:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114440320:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766121.483642:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766121.483676:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766121/real 1775766121]  req@ffff9f14852d2a00 x1862022114440320/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766146 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766121.483701:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2a00 x1862022114440320/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766146 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766121.483729:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766121.483734:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766121.483738:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114440320:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775766121.483749:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114440448:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766121.483783:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766121.483802:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766121/real 1775766121]  req@ffff9f14852d2680 x1862022114440448/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766146 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766121.483869:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2680 x1862022114440448/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766146 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766121.483880:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766121.483885:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766121.483923:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114440448:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775766121.483971:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114440576:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766121.484000:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766121.484015:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766121/real 1775766121]  req@ffff9f14852d1180 x1862022114440576/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766146 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766121.484032:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1180 x1862022114440576/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766146 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766121.484042:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766121.484046:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766121.484050:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114440576:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766121.484057:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114440704:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766121.484083:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766121.484098:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766121/real 1775766121]  req@ffff9f14852d2d80 x1862022114440704/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766146 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766121.484111:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2d80 x1862022114440704/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766146 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766121.484120:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766121.484123:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766121.484126:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114440704:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766121.484132:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114440832:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766121.484149:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766121.484160:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766121/real 1775766121]  req@ffff9f14852d0700 x1862022114440832/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766146 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766121.484173:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0700 x1862022114440832/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766146 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766121.484182:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766121.484185:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766121.484189:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114440832:192.168.202.110@tcp:8:kworker.0
00010000:00010000:1.0:1775766121.738783:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775766122.506639:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766122.762761:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775766123.530677:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775766123.786740:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775766124.554718:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775766124.810809:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775766125.578753:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766125.834709:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775766126.602631:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:0.0:1775766126.602702:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766126.602727:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1775766126.602734:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766126.602749:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3504
00000100:00080000:0.0:1775766126.602797:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766126.602806:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775766126.602857:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d1180 x1862022114440960/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775766126.602892:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766126.602896:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775766126.602898:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766126.602903:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3504
00000100:00080000:0.0:1775766126.602907:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766126.602911:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775766126.602923:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775766126.602936:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3480 x1862022114441088/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775766126.602936:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114440960:192.168.202.110@tcp:250:kworker.0
00000100:00080000:0.0:1775766126.602968:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766126.602973:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:0.0:1775766126.602975:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766126.602983:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3504
00000100:00080000:0.0:1775766126.602989:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766126.602994:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775766126.603002:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775766126.603019:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0000 x1862022114441216/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:3.0:1775766126.603027:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:0.0:1775766126.603056:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766126.603062:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775766126.603064:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766126.603071:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3504
00000100:00080000:0.0:1775766126.603077:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00100000:3.0:1775766126.603082:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766126/real 1775766126]  req@ffff9f14a04d1180 x1862022114440960/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766151 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766126.603086:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775766126.603103:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766126.603114:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1180 x1862022114440960/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766151 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766126.603121:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d1500 x1862022114441344/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766126.603153:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766126.603155:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766126.603162:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775766126.603162:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:0.0:1775766126.603168:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766126.603175:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3504
00000100:00080000:0.0:1775766126.603179:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766126.603184:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775766126.603187:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114440960:192.168.202.110@tcp:250:kworker.0
00000008:00100000:0.0:1775766126.603190:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775766126.603204:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3800 x1862022114441472/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775766126.603213:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766126.603216:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775766126.603218:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766126.603225:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3504
00000100:00100000:3.0:1775766126.603226:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114441088:192.168.202.110@tcp:38:kworker.0
00000100:00080000:0.0:1775766126.603231:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766126.603239:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775766126.603244:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775766126.603256:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d1180 x1862022114441600/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775766126.603269:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766126.603278:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000800:00000100:3.0:1775766126.603279:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:0.0:1775766126.603281:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766126.603288:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3504
00000100:00080000:0.0:1775766126.603295:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766126.603305:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775766126.603306:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766126/real 1775766126]  req@ffff9f14a04d3480 x1862022114441088/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766151 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:0.0:1775766126.603318:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775766126.603341:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d2300 x1862022114441728/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775766126.603341:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3480 x1862022114441088/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766151 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766126.603419:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766126.603435:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00080000:0.0:1775766126.603445:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:3.0:1775766126.603450:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114441088:192.168.202.110@tcp:38:kworker.0
00000100:00080000:0.0:1775766126.603453:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775766126.603460:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766126.603467:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3504
00000100:00100000:3.0:1775766126.603469:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114441216:192.168.202.110@tcp:38:kworker.0
00000100:00080000:0.0:1775766126.603472:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766126.603484:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775766126.603491:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:3.0:1775766126.603507:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766126.603508:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0e00 x1862022114441856/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775766126.603519:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766126/real 1775766126]  req@ffff9f14a04d0000 x1862022114441216/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766151 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766126.603535:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0000 x1862022114441216/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766151 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766126.603541:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766126.603544:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766126.603547:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114441216:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766126.603573:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114441344:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775766126.603588:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766126.603595:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766126/real 1775766126]  req@ffff9f14a04d1500 x1862022114441344/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766151 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766126.603603:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1500 x1862022114441344/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766151 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766126.603608:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766126.603610:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766126.603613:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114441344:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775766126.603617:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114441472:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775766126.603627:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766126.603632:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766126/real 1775766126]  req@ffff9f14a04d3800 x1862022114441472/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766151 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766126.603639:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3800 x1862022114441472/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766151 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766126.603649:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766126.603651:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766126.603653:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114441472:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775766126.603667:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114441600:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775766126.603677:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766126.603683:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766126/real 1775766126]  req@ffff9f14a04d1180 x1862022114441600/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766151 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766126.603699:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1180 x1862022114441600/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766151 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766126.603708:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766126.603711:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766126.603714:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114441600:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766126.603721:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114441728:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775766126.603742:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766126.603753:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766126/real 1775766126]  req@ffff9f14a04d2300 x1862022114441728/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766151 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766126.603776:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d2300 x1862022114441728/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766151 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766126.603792:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766126.603794:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766126.603797:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114441728:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766126.603801:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114441856:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775766126.603816:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766126.603823:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766126/real 1775766126]  req@ffff9f14a04d0e00 x1862022114441856/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766151 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766126.603831:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0e00 x1862022114441856/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766151 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766126.603836:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766126.603838:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766126.603840:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114441856:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775766126.858719:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775766127.626618:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766127.882696:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775766128.650715:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775766128.906853:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775766129.674617:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766129.930759:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775766130.698704:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775766130.954779:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775766131.722705:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775766131.722750:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766131.722758:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775766131.722763:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766131.722773:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3509
00000100:00080000:3.0:1775766131.722785:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766131.722793:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775766131.722831:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745880 x1862022114441984/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766131.722866:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766131.722870:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775766131.722872:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766131.722879:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3509
00000100:00080000:3.0:1775766131.722884:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766131.722889:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766131.722900:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775766131.722913:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114441984:192.168.202.110@tcp:250:kworker.0
00000100:00100000:3.0:1775766131.722921:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746680 x1862022114442112/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766131.722933:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766131.722935:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775766131.722937:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766131.722942:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3509
00000100:00080000:3.0:1775766131.722945:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766131.722949:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766131.722953:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766131.722965:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747b80 x1862022114442240/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766131.722973:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766131.722976:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775766131.722977:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766131.722979:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3509
00000100:00080000:3.0:1775766131.722982:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766131.722984:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766131.722988:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766131.722995:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744e00 x1862022114442368/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766131.723002:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766131.723004:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775766131.723005:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766131.723007:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3509
00000100:00080000:3.0:1775766131.723010:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766131.723012:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766131.723015:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766131.723025:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744380 x1862022114442496/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766131.723030:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766131.723033:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775766131.723035:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766131.723039:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3509
00000100:00080000:3.0:1775766131.723043:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766131.723046:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766131.723050:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766131.723057:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746a00 x1862022114442624/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766131.723063:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766131.723065:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775766131.723066:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766131.723068:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3509
00000100:00080000:3.0:1775766131.723073:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766131.723077:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766131.723082:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766131.723095:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746300 x1862022114442752/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766131.723104:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766131.723107:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775766131.723109:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766131.723113:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3509
00000100:00080000:3.0:1775766131.723117:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766131.723120:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766131.723125:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766131.723137:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747480 x1862022114442880/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775766131.723192:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114442112:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766131.723265:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114442240:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766131.723280:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114442368:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775766131.723293:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114442496:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775766131.723304:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114442624:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766131.723314:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114442752:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766131.723325:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114442880:192.168.202.110@tcp:8:kworker.0
00010000:00010000:3.0:1775766131.978761:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775766132.746685:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766133.002750:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775766133.770649:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766134.026765:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775766134.794705:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000400:00000100:2.0:1775766134.794935:0:2391:0:(lib-socket.c:454:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.202.110:988
00000400:00000100:2.0:1775766134.794981:0:2391:0:(acceptor.c:111:lnet_connect_console_error()) Connection to 192.168.202.110@tcp at host 192.168.202.110 was unreachable: the network or that node may be down, or Lustre may be misconfigured.
00000800:00000100:2.0:1775766134.795009:0:2391:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:2.0:1775766134.795049:0:2391:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:2.0:1775766134.795057:0:2391:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:2.0:1775766134.795063:0:2391:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:2.0:1775766134.795070:0:2391:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:2.0:1775766134.795075:0:2391:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:2.0:1775766134.795081:0:2391:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:2.0:1775766134.795086:0:2391:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000100:00100000:0.0:1775766134.795138:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766131/real 1775766134]  req@ffff9f1485746680 x1862022114442112/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766156 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766134.795178:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746680 x1862022114442112/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766156 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766134.795195:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766134.795201:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766134.795206:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114442112:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766134.795217:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766131/real 1775766134]  req@ffff9f1485747b80 x1862022114442240/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766156 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766134.795230:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747b80 x1862022114442240/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766156 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766134.795238:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766134.795242:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766134.795246:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114442240:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766134.795255:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766131/real 1775766134]  req@ffff9f1485744e00 x1862022114442368/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766156 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766134.795267:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744e00 x1862022114442368/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766156 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766134.795274:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766134.795278:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766134.795282:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114442368:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775766134.795289:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766131/real 1775766134]  req@ffff9f1485744380 x1862022114442496/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766156 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766134.795302:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744380 x1862022114442496/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766156 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766134.795312:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766134.795315:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766134.795319:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114442496:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775766134.795328:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766131/real 1775766134]  req@ffff9f1485746a00 x1862022114442624/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766156 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766134.795340:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746a00 x1862022114442624/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766156 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766134.795356:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766134.795360:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766134.795364:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114442624:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766134.795372:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766131/real 1775766134]  req@ffff9f1485746300 x1862022114442752/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766156 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766134.795385:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746300 x1862022114442752/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766156 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766134.795394:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766134.795398:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766134.795402:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114442752:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766134.795410:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766131/real 1775766134]  req@ffff9f1485747480 x1862022114442880/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766156 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766134.795424:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747480 x1862022114442880/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766156 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766134.795433:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766134.795436:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766134.795440:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114442880:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775766134.795449:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766131/real 1775766134]  req@ffff9f1485745880 x1862022114441984/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766156 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766134.795464:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745880 x1862022114441984/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766156 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766134.795471:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766134.795476:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766134.795479:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114441984:192.168.202.110@tcp:250:kworker.0
00010000:00010000:2.0:1775766135.050746:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775766135.818682:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775766136.074715:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1775766136.842622:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:1.0:1775766136.842628:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775766136.842630:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775766136.842634:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766136.842674:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3514
00000100:00080000:3.0:1775766136.842684:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766136.842697:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775766136.842727:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744a80 x1862022114443008/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766136.842748:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766136.842751:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775766136.842752:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766136.842755:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3514
00000100:00080000:3.0:1775766136.842759:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766136.842763:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766136.842774:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775766136.842787:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114443008:192.168.202.110@tcp:250:kworker.0
00000100:00100000:3.0:1775766136.842789:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745880 x1862022114443136/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766136.842804:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766136.842808:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775766136.842810:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766136.842815:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3514
00000100:00080000:3.0:1775766136.842819:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766136.842823:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766136.842827:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766136.842839:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747480 x1862022114443264/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1775766136.842850:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775766136.842852:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766136.842856:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775766136.842858:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766136.842864:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3514
00000100:00080000:3.0:1775766136.842868:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766136.842873:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766136.842880:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775766136.842893:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766136/real 1775766136]  req@ffff9f1485744a80 x1862022114443008/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766161 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766136.842895:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746300 x1862022114443392/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766136.842936:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766136.842938:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775766136.842940:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775766136.842942:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744a80 x1862022114443008/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766161 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766136.842945:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3514
00000100:00080000:3.0:1775766136.842951:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766136.842960:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00080000:0.0:1775766136.842964:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000008:00100000:3.0:1775766136.842966:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00080000:0.0:1775766136.842972:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766136.842977:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114443008:192.168.202.110@tcp:250:kworker.0
00000100:00100000:3.0:1775766136.842983:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746a00 x1862022114443520/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766136.842996:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766136.843000:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775766136.843002:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775766136.843007:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114443136:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775766136.843010:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3514
00000100:00080000:3.0:1775766136.843018:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766136.843023:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766136.843029:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766136.843044:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744380 x1862022114443648/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1775766136.843045:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775766136.843058:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766136.843063:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775766136.843065:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775766136.843072:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766136/real 1775766136]  req@ffff9f1485745880 x1862022114443136/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766161 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766136.843072:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3514
00000100:00080000:3.0:1775766136.843079:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766136.843087:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766136.843095:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775766136.843097:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745880 x1862022114443136/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766161 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766136.843109:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744e00 x1862022114443776/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766136.843119:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766136.843123:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766136.843123:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775766136.843126:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766136.843132:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00080000:3.0:1775766136.843133:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3514
00000100:00080000:3.0:1775766136.843140:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1775766136.843141:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114443136:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775766136.843148:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775766136.843153:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114443264:192.168.202.110@tcp:38:kworker.0
00000008:00100000:3.0:1775766136.843153:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766136.843172:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747b80 x1862022114443904/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1775766136.843187:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766136.843201:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766136/real 1775766136]  req@ffff9f1485747480 x1862022114443264/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766161 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766136.843214:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747480 x1862022114443264/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766161 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766136.843222:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766136.843226:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766136.843228:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114443264:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766136.843235:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114443392:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766136.843259:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766136.843270:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766136/real 1775766136]  req@ffff9f1485746300 x1862022114443392/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766161 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766136.843282:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746300 x1862022114443392/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766161 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766136.843291:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766136.843294:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766136.843297:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114443392:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775766136.843303:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114443520:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766136.843323:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766136.843333:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766136/real 1775766136]  req@ffff9f1485746a00 x1862022114443520/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766161 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766136.843346:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746a00 x1862022114443520/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766161 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766136.843354:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766136.843358:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766136.843361:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114443520:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775766136.843380:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114443648:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766136.843402:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766136.843413:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766136/real 1775766136]  req@ffff9f1485744380 x1862022114443648/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766161 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766136.843425:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744380 x1862022114443648/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766161 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766136.843441:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766136.843445:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766136.843448:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114443648:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766136.843455:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114443776:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766136.843476:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766136.843486:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766136/real 1775766136]  req@ffff9f1485744e00 x1862022114443776/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766161 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766136.843499:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744e00 x1862022114443776/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766161 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766136.843507:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766136.843510:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766136.843513:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114443776:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766136.843519:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114443904:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766136.843538:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766136.843598:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766136/real 1775766136]  req@ffff9f1485747b80 x1862022114443904/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766161 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766136.843613:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747b80 x1862022114443904/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766161 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766136.843622:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766136.843625:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766136.843629:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114443904:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775766137.098697:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775766137.866768:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775766138.122759:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775766138.890665:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775766139.146869:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775766139.914682:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766140.170795:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775766140.938679:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766141.194720:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775766141.962635:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775766141.962635:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766141.962645:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775766141.962650:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766141.962658:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3519
00000100:00080000:3.0:1775766141.962668:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766141.962673:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775766141.962713:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746680 x1862022114444032/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766141.962735:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766141.962738:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775766141.962739:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766141.962742:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3519
00000100:00080000:3.0:1775766141.962745:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766141.962747:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766141.962755:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766141.962763:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744700 x1862022114444160/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766141.962772:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1775766141.962773:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114444032:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775766141.962775:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775766141.962778:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766141.962781:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3519
00000100:00080000:3.0:1775766141.962783:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766141.962788:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766141.962791:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766141.962798:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745500 x1862022114444288/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766141.962804:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766141.962805:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775766141.962807:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766141.962809:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3519
00000100:00080000:3.0:1775766141.962812:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766141.962814:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766141.962818:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766141.962824:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744000 x1862022114444416/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766141.962830:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766141.962831:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775766141.962833:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766141.962835:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3519
00000100:00080000:3.0:1775766141.962838:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000800:00000100:0.0:1775766141.962842:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775766141.962845:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766141.962849:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766141.962855:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747800 x1862022114444544/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766141.962861:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766141.962863:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775766141.962864:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766141.962866:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3519
00000100:00080000:3.0:1775766141.962869:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766141.962872:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766141.962876:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775766141.962879:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766141/real 1775766141]  req@ffff9f1485746680 x1862022114444032/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766166 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766141.962886:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745c00 x1862022114444672/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766141.962896:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1775766141.962899:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746680 x1862022114444032/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766166 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766141.962900:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775766141.962903:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766141.962910:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3519
00000100:00080000:3.0:1775766141.962915:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766141.962918:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766141.962918:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766141.962921:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00080000:0.0:1775766141.962925:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766141.962929:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114444032:192.168.202.110@tcp:250:kworker.0
00000100:00100000:3.0:1775766141.962931:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745180 x1862022114444800/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766141.962939:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766141.962941:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775766141.962942:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766141.962945:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3519
00000100:00080000:3.0:1775766141.962947:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766141.962950:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775766141.962953:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114444160:192.168.202.110@tcp:38:kworker.0
00000008:00100000:3.0:1775766141.962955:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766141.962965:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745f80 x1862022114444928/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1775766141.962975:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766141.962994:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766141/real 1775766141]  req@ffff9f1485744700 x1862022114444160/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766166 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766141.963003:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744700 x1862022114444160/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766166 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766141.963008:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766141.963011:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766141.963013:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114444160:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766141.963017:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114444288:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766141.963029:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766141.963037:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766141/real 1775766141]  req@ffff9f1485745500 x1862022114444288/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766166 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766141.963049:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745500 x1862022114444288/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766166 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766141.963058:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766141.963061:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766141.963064:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114444288:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766141.963071:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114444416:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766141.963091:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766141.963102:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766141/real 1775766141]  req@ffff9f1485744000 x1862022114444416/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766166 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766141.963113:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744000 x1862022114444416/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766166 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766141.963119:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766141.963121:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766141.963123:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114444416:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775766141.963127:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114444544:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766141.963141:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766141.963147:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766141/real 1775766141]  req@ffff9f1485747800 x1862022114444544/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766166 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766141.963154:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747800 x1862022114444544/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766166 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766141.963160:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766141.963162:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766141.963164:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114444544:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775766141.963167:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114444672:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766141.963215:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766141.963223:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766141/real 1775766141]  req@ffff9f1485745c00 x1862022114444672/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766166 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766141.963235:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745c00 x1862022114444672/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766166 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766141.963243:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766141.963247:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766141.963250:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114444672:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766141.963257:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114444800:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766141.963277:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766141.963287:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766141/real 1775766141]  req@ffff9f1485745180 x1862022114444800/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766166 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766141.963306:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745180 x1862022114444800/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766166 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766141.963311:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766141.963313:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766141.963316:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114444800:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766141.963330:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114444928:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766141.963345:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766141.963351:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766141/real 1775766141]  req@ffff9f1485745f80 x1862022114444928/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766166 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766141.963359:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745f80 x1862022114444928/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766166 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766141.963364:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766141.963365:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766141.963367:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114444928:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775766142.218709:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775766142.986693:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775766143.242689:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775766144.010703:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766144.266879:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775766145.034649:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775766145.290715:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775766146.058651:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775766146.314731:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1775766147.082652:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:3.0:1775766147.082656:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:2.0:1775766147.082661:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1775766147.082666:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766147.082673:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3524
00000100:00080000:2.0:1775766147.082681:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766147.082687:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775766147.082727:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3800 x1862022114445056/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766147.082747:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766147.082749:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775766147.082751:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766147.082754:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3524
00000100:00080000:2.0:1775766147.082756:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766147.082758:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766147.082766:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766147.082775:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0380 x1862022114445184/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766147.082782:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1775766147.082785:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114445056:192.168.202.110@tcp:250:kworker.0
00000100:00080000:2.0:1775766147.082786:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775766147.082789:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766147.082795:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3524
00000100:00080000:2.0:1775766147.082797:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766147.082799:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766147.082803:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766147.082812:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0000 x1862022114445312/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766147.082819:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766147.082821:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775766147.082822:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766147.082825:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3524
00000100:00080000:2.0:1775766147.082827:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766147.082829:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766147.082832:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775766147.082838:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3b80 x1862022114445440/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766147.082843:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766147.082846:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1775766147.082848:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000800:00000100:0.0:1775766147.082850:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:2.0:1775766147.082853:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3524
00000100:00080000:2.0:1775766147.082856:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766147.082858:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766147.082862:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775766147.082869:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0e00 x1862022114445568/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766147.082874:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766147.082876:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775766147.082877:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766147.082879:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3524
00000100:00080000:2.0:1775766147.082882:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1775766147.082944:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766147/real 1775766147]  req@ffff9f14852d3800 x1862022114445056/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766172 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766147.082959:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3800 x1862022114445056/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766172 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766147.082967:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766147.082970:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766147.082983:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114445056:192.168.202.110@tcp:250:kworker.0
00000100:00100000:0.0:1775766147.083001:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114445184:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766147.083029:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766147.083054:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766147/real 1775766147]  req@ffff9f14852d0380 x1862022114445184/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766172 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766147.083068:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0380 x1862022114445184/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766172 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766147.083077:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766147.083080:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766147.083083:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114445184:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766147.083088:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114445312:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766147.083103:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766147.083141:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766147/real 1775766147]  req@ffff9f14852d0000 x1862022114445312/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766172 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766147.083150:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0000 x1862022114445312/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766172 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766147.083157:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766147.083160:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766147.083163:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114445312:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766147.083168:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114445440:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766147.083181:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766147.083186:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766147/real 1775766147]  req@ffff9f14852d3b80 x1862022114445440/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766172 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766147.083194:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3b80 x1862022114445440/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766172 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766147.083210:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766147.083213:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766147.083217:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114445440:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775766147.083221:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114445568:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766147.083230:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766147.083235:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766147/real 1775766147]  req@ffff9f14852d0e00 x1862022114445568/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766172 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766147.083249:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0e00 x1862022114445568/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766172 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766147.083254:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766147.083257:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766147.083258:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114445568:192.168.202.110@tcp:8:kworker.0
00000100:00080000:2.0:1775766147.084118:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766147.084123:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766147.084137:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1880 x1862022114445696/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766147.084151:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766147.084153:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775766147.084155:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766147.084157:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3524
00000100:00080000:2.0:1775766147.084160:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766147.084163:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766147.084167:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766147.084173:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3100 x1862022114445824/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766147.084181:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1775766147.084185:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114445696:192.168.202.110@tcp:38:kworker.0
00000100:00080000:2.0:1775766147.084186:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775766147.084189:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766147.084196:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3524
00000100:00080000:2.0:1775766147.084198:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766147.084200:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766147.084213:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775766147.084222:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2300 x1862022114445952/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1775766147.084229:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766147.084245:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766147/real 1775766147]  req@ffff9f14852d1880 x1862022114445696/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766172 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766147.084255:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1880 x1862022114445696/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766172 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766147.084260:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766147.084262:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766147.084265:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114445696:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766147.084283:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114445824:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766147.084294:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766147.084300:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766147/real 1775766147]  req@ffff9f14852d3100 x1862022114445824/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766172 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766147.084307:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3100 x1862022114445824/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766172 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766147.084312:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766147.084313:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766147.084315:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114445824:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766147.084318:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114445952:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766147.084327:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766147.084332:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766147/real 1775766147]  req@ffff9f14852d2300 x1862022114445952/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766172 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766147.084340:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2300 x1862022114445952/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766172 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766147.084345:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766147.084347:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766147.084348:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114445952:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775766147.338707:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766148.106689:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766148.362781:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766149.130675:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766149.386734:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766150.154637:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775766150.410709:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766151.178704:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766151.434687:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766152.202647:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:1.0:1775766152.202651:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766152.202658:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1775766152.202661:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766152.202669:0:8:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3529
00000100:00080000:1.0:1775766152.202679:0:8:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766152.202684:0:8:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775766152.202716:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdf80 x1862022114446080/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766152.202738:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766152.202741:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775766152.202742:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766152.202745:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3529
00000100:00080000:1.0:1775766152.202748:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766152.202750:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766152.202769:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775766152.202777:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ffb80 x1862022114446208/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766152.202784:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766152.202787:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775766152.202789:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766152.202791:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3529
00000100:00080000:1.0:1775766152.202793:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766152.202795:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766152.202799:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775766152.202806:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114446080:192.168.202.110@tcp:250:kworker.0
00000100:00100000:1.0:1775766152.202807:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff800 x1862022114446336/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766152.202820:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766152.202822:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775766152.202824:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766152.202827:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3529
00000100:00080000:1.0:1775766152.202828:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766152.202831:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766152.202833:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775766152.202844:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdc00 x1862022114446464/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766152.202853:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766152.202856:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1775766152.202858:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766152.202861:0:8:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3529
00000100:00080000:1.0:1775766152.202878:0:8:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766152.202881:0:8:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000800:00000100:0.0:1775766152.202887:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000008:00100000:1.0:1775766152.202888:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775766152.202903:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fed80 x1862022114446592/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766152.202914:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766152.202918:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775766152.202920:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766152.202925:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3529
00000100:00080000:1.0:1775766152.202930:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1775766152.202931:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766152/real 1775766152]  req@ffff9f14ab4fdf80 x1862022114446080/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766177 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766152.202939:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766152.202949:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775766152.202960:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdf80 x1862022114446080/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766177 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775766152.202964:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fca80 x1862022114446720/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775766152.202985:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775766152.202986:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766152.202992:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775766152.202994:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766152.202997:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:1.0:1775766152.203000:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3529
00000100:00080000:1.0:1775766152.203005:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1775766152.203007:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114446080:192.168.202.110@tcp:250:kworker.0
00000100:00080000:1.0:1775766152.203011:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766152.203018:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775766152.203022:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ffb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114446208:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775766152.203028:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fce00 x1862022114446848/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766152.203035:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766152.203037:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775766152.203039:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766152.203041:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3529
00000100:00080000:1.0:1775766152.203044:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766152.203049:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000800:00000100:0.0:1775766152.203051:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000008:00100000:1.0:1775766152.203062:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775766152.203067:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766152/real 1775766152]  req@ffff9f14ab4ffb80 x1862022114446208/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766177 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775766152.203080:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff100 x1862022114446976/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775766152.203085:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ffb80 x1862022114446208/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766177 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766152.203107:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766152.203112:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766152.203116:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ffb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114446208:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766152.203140:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114446336:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766152.203158:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766152.203167:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766152/real 1775766152]  req@ffff9f14ab4ff800 x1862022114446336/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766177 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766152.203179:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff800 x1862022114446336/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766177 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766152.203194:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766152.203198:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766152.203201:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114446336:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766152.203208:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114446464:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766152.203224:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766152.203233:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766152/real 1775766152]  req@ffff9f14ab4fdc00 x1862022114446464/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766177 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766152.203244:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdc00 x1862022114446464/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766177 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766152.203253:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766152.203256:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766152.203259:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114446464:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775766152.203265:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114446592:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766152.203279:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766152.203287:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766152/real 1775766152]  req@ffff9f14ab4fed80 x1862022114446592/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766177 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766152.203308:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fed80 x1862022114446592/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766177 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766152.203317:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766152.203320:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766152.203324:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114446592:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775766152.203330:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114446720:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766152.203347:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766152.203357:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766152/real 1775766152]  req@ffff9f14ab4fca80 x1862022114446720/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766177 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766152.203368:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fca80 x1862022114446720/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766177 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766152.203377:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766152.203380:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766152.203383:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114446720:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766152.203389:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114446848:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766152.203414:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766152.203424:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766152/real 1775766152]  req@ffff9f14ab4fce00 x1862022114446848/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766177 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766152.203437:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fce00 x1862022114446848/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766177 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766152.203445:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766152.203448:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766152.203461:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114446848:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766152.203467:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114446976:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766152.203482:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766152.203491:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766152/real 1775766152]  req@ffff9f14ab4ff100 x1862022114446976/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766177 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766152.203503:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff100 x1862022114446976/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766177 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766152.203527:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766152.203530:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766152.203534:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114446976:192.168.202.110@tcp:8:kworker.0
00010000:00010000:3.0:1775766152.458703:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766153.226638:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775766153.482781:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766154.250668:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766154.506790:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766155.274641:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766155.530709:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766156.298655:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766156.554673:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766157.322659:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:1.0:1775766157.322685:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766157.322694:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1775766157.322698:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766157.322718:0:8:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3534
00000100:00080000:1.0:1775766157.322728:0:8:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766157.322733:0:8:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775766157.322766:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fd880 x1862022114447104/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766157.322795:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766157.322799:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775766157.322801:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766157.322806:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3534
00000100:00080000:1.0:1775766157.322810:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766157.322814:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766157.322834:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775766157.322848:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114447104:192.168.202.110@tcp:250:kworker.0
00000100:00100000:1.0:1775766157.322853:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fe300 x1862022114447232/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766157.322865:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766157.322868:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775766157.322869:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766157.322872:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3534
00000100:00080000:1.0:1775766157.322875:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766157.322879:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766157.322890:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775766157.322899:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fc380 x1862022114447360/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766157.322906:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766157.322909:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775766157.322910:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766157.322912:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3534
00000100:00080000:1.0:1775766157.322915:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000800:00000100:0.0:1775766157.322919:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:1.0:1775766157.322919:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766157.322926:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775766157.322934:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fea00 x1862022114447488/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766157.322941:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766157.322943:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1775766157.322944:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766157.322946:0:8:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3534
00000100:00080000:1.0:1775766157.322949:0:8:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766157.322951:0:8:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766157.322954:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775766157.322964:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766157/real 1775766157]  req@ffff9f14ab4fd880 x1862022114447104/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766182 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775766157.322965:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff100 x1862022114447616/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766157.322997:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766157.323000:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775766157.323001:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766157.323005:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3534
00000100:00100000:0.0:1775766157.323008:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fd880 x1862022114447104/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766182 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766157.323009:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766157.323016:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766157.323023:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00080000:0.0:1775766157.323026:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766157.323031:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766157.323037:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114447104:192.168.202.110@tcp:250:kworker.0
00000100:00100000:1.0:1775766157.323037:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fce00 x1862022114447744/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766157.323053:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766157.323056:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775766157.323058:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775766157.323064:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114447232:192.168.202.110@tcp:38:kworker.0
00000100:00080000:1.0:1775766157.323065:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3534
00000100:00080000:1.0:1775766157.323073:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766157.323085:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766157.323091:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775766157.323101:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fca80 x1862022114447872/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1775766157.323108:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:1.0:1775766157.323114:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766157.323118:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775766157.323119:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766157.323123:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3534
00000100:00080000:1.0:1775766157.323127:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1775766157.323130:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766157/real 1775766157]  req@ffff9f14ab4fe300 x1862022114447232/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766182 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766157.323131:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775766157.323148:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fe300 x1862022114447232/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766182 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766157.323160:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766157.323162:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766157.323165:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114447232:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766157.323171:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114447360:192.168.202.110@tcp:38:kworker.0
00000008:00100000:1.0:1775766157.323176:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775766157.323192:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fed80 x1862022114448000/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1775766157.323195:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766157.323206:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766157/real 1775766157]  req@ffff9f14ab4fc380 x1862022114447360/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766182 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766157.323217:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fc380 x1862022114447360/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766182 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766157.323222:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766157.323225:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766157.323227:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114447360:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766157.323231:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114447488:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766157.323246:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766157.323252:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766157/real 1775766157]  req@ffff9f14ab4fea00 x1862022114447488/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766182 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766157.323260:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fea00 x1862022114447488/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766182 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766157.323266:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766157.323268:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766157.323271:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114447488:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775766157.323275:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114447616:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766157.323288:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766157.323294:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766157/real 1775766157]  req@ffff9f14ab4ff100 x1862022114447616/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766182 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766157.323303:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff100 x1862022114447616/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766182 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766157.323309:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766157.323310:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766157.323312:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114447616:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775766157.323316:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114447744:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766157.323328:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766157.323339:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766157/real 1775766157]  req@ffff9f14ab4fce00 x1862022114447744/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766182 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766157.323358:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fce00 x1862022114447744/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766182 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766157.323363:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766157.323366:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766157.323368:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114447744:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766157.323391:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114447872:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766157.323408:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766157.323417:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766157/real 1775766157]  req@ffff9f14ab4fca80 x1862022114447872/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766182 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766157.323437:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fca80 x1862022114447872/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766182 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766157.323442:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766157.323444:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766157.323446:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114447872:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766157.323450:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114448000:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766157.323485:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766157.323512:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766157/real 1775766157]  req@ffff9f14ab4fed80 x1862022114448000/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766182 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766157.323527:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fed80 x1862022114448000/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766182 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766157.323537:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766157.323542:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766157.323547:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114448000:192.168.202.110@tcp:8:kworker.0
00010000:00010000:1.0:1775766157.578744:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766158.346614:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766158.602720:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766159.370642:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775766159.626729:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766160.394627:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775766160.650830:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766161.418673:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766161.674831:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766162.442685:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:1.0:1775766162.442687:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766162.442697:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1775766162.442702:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766162.442713:0:8:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3539
00000100:00080000:1.0:1775766162.442725:0:8:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766162.442734:0:8:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775766162.442792:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdc00 x1862022114448128/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766162.442819:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766162.442824:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775766162.442826:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766162.442831:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3539
00000100:00080000:1.0:1775766162.442835:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766162.442840:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766162.442850:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775766162.442877:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff800 x1862022114448256/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775766162.442888:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114448128:192.168.202.110@tcp:250:kworker.0
00000100:00080000:1.0:1775766162.442892:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766162.442897:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775766162.442899:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766162.442905:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3539
00000100:00080000:1.0:1775766162.442910:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766162.442914:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766162.442921:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775766162.442933:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ffb80 x1862022114448384/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766162.442945:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766162.442949:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000800:00000100:0.0:1775766162.442951:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:1.0:1775766162.442953:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766162.442960:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3539
00000100:00080000:1.0:1775766162.442965:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766162.442971:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766162.442988:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775766162.442991:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766162/real 1775766162]  req@ffff9f14ab4fdc00 x1862022114448128/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766187 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775766162.443007:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdf80 x1862022114448512/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775766162.443010:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdc00 x1862022114448128/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766187 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766162.443041:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775766162.443042:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766162.443047:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:1.0:1775766162.443050:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00100000:0.0:1775766162.443054:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114448128:192.168.202.110@tcp:250:kworker.0
00000100:00080000:1.0:1775766162.443055:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766162.443065:0:8:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3539
00000100:00080000:1.0:1775766162.443071:0:8:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766162.443076:0:8:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775766162.443079:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114448256:192.168.202.110@tcp:38:kworker.0
00000008:00100000:1.0:1775766162.443084:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:0.0:1775766162.443108:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775766162.443112:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdc00 x1862022114448640/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775766162.443125:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766162/real 1775766162]  req@ffff9f14ab4ff800 x1862022114448256/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766187 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766162.443129:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766162.443139:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775766162.443143:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766162.443151:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3539
00000100:00100000:0.0:1775766162.443155:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff800 x1862022114448256/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766187 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766162.443158:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766162.443167:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00080000:0.0:1775766162.443172:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000008:00100000:1.0:1775766162.443176:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00080000:0.0:1775766162.443178:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766162.443183:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114448256:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766162.443189:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ffb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114448384:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775766162.443194:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fed80 x1862022114448768/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766162.443207:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000800:00000100:0.0:1775766162.443210:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:1.0:1775766162.443216:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775766162.443219:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775766162.443226:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766162/real 1775766162]  req@ffff9f14ab4ffb80 x1862022114448384/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766187 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766162.443226:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3539
00000100:00080000:1.0:1775766162.443234:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1775766162.443241:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ffb80 x1862022114448384/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766187 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766162.443243:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00080000:0.0:1775766162.443252:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000008:00100000:1.0:1775766162.443254:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00080000:0.0:1775766162.443258:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766162.443261:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ffb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114448384:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766162.443268:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114448512:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775766162.443273:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fca80 x1862022114448896/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1775766162.443286:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:1.0:1775766162.443297:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1775766162.443298:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766162/real 1775766162]  req@ffff9f14ab4fdf80 x1862022114448512/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766187 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766162.443306:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775766162.443309:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766162.443319:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3539
00000100:00080000:1.0:1775766162.443326:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1775766162.443330:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdf80 x1862022114448512/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766187 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766162.443334:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766162.443344:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00080000:0.0:1775766162.443347:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766162.443354:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766162.443358:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114448512:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775766162.443363:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fce00 x1862022114449024/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775766162.443377:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114448640:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766162.443396:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766162.443432:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766162/real 1775766162]  req@ffff9f14ab4fdc00 x1862022114448640/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766187 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766162.443446:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdc00 x1862022114448640/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766187 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766162.443455:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766162.443458:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766162.443462:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114448640:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775766162.443469:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114448768:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766162.443490:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766162.443499:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766162/real 1775766162]  req@ffff9f14ab4fed80 x1862022114448768/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766187 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766162.443513:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fed80 x1862022114448768/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766187 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766162.443522:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766162.443526:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766162.443529:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114448768:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766162.443536:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114448896:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766162.443590:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766162.443601:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766162/real 1775766162]  req@ffff9f14ab4fca80 x1862022114448896/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766187 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766162.443610:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fca80 x1862022114448896/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766187 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766162.443616:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766162.443619:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766162.443621:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114448896:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766162.443632:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114449024:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766162.443646:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766162.443652:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766162/real 1775766162]  req@ffff9f14ab4fce00 x1862022114449024/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766187 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766162.443676:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fce00 x1862022114449024/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766187 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766162.443687:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766162.443689:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766162.443691:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114449024:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775766162.698713:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766163.466611:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766163.722698:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766164.490661:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775766164.746731:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766165.514634:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766165.770875:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766166.538665:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775766166.794717:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766167.562649:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:0.0:1775766167.562701:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766167.562713:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1775766167.562719:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766167.562731:0:8:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3545
00000100:00080000:0.0:1775766167.562746:0:8:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766167.562754:0:8:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775766167.562823:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0a80 x1862022114449152/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775766167.562867:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766167.562870:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775766167.562872:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766167.562876:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3545
00000100:00080000:0.0:1775766167.562879:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766167.562881:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775766167.562892:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775766167.562901:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3b80 x1862022114449280/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775766167.562908:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766167.562911:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:0.0:1775766167.562913:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766167.562916:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3545
00000100:00080000:0.0:1775766167.562919:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766167.562922:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775766167.562928:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775766167.562934:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d1f80 x1862022114449408/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775766167.562943:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766167.562945:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775766167.562948:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766167.562951:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3545
00000100:00080000:0.0:1775766167.562956:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00100000:3.0:1775766167.562959:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114449152:192.168.202.110@tcp:250:kworker.0
00000100:00080000:0.0:1775766167.562963:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775766167.562973:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775766167.562981:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0380 x1862022114449536/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775766167.562991:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766167.562994:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1775766167.562995:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766167.562998:0:8:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3545
00000100:00080000:0.0:1775766167.563000:0:8:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766167.563003:0:8:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775766167.563013:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775766167.563022:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d2a00 x1862022114449664/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775766167.563029:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766167.563032:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775766167.563033:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766167.563046:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3545
00000100:00080000:0.0:1775766167.563049:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766167.563052:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775766167.563056:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000800:00000100:3.0:1775766167.563060:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766167.563072:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3100 x1862022114449792/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775766167.563084:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766167.563087:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:0.0:1775766167.563089:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766167.563094:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3545
00000100:00080000:0.0:1775766167.563097:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766167.563099:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775766167.563103:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775766167.563115:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0e00 x1862022114449920/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775766167.563123:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766167.563126:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775766167.563130:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:3.0:1775766167.563134:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766167/real 1775766167]  req@ffff9f14a04d0a80 x1862022114449152/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766192 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766167.563139:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3545
00000100:00080000:0.0:1775766167.563151:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766167.563167:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775766167.563192:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775766167.563212:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d2300 x1862022114450048/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775766167.563219:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0a80 x1862022114449152/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766192 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766167.563254:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766167.563261:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766167.563265:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114449152:192.168.202.110@tcp:250:kworker.0
00000100:00100000:3.0:1775766167.563272:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114449280:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775766167.563290:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766167.563299:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766167/real 1775766167]  req@ffff9f14a04d3b80 x1862022114449280/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766192 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766167.563307:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3b80 x1862022114449280/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766192 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766167.563313:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766167.563316:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766167.563319:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114449280:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766167.563323:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114449408:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775766167.563334:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766167.563340:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766167/real 1775766167]  req@ffff9f14a04d1f80 x1862022114449408/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766192 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766167.563347:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1f80 x1862022114449408/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766192 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766167.563353:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766167.563355:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766167.563357:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114449408:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766167.563403:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114449536:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775766167.563417:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766167.563425:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766167/real 1775766167]  req@ffff9f14a04d0380 x1862022114449536/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766192 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766167.563433:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0380 x1862022114449536/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766192 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766167.563439:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766167.563441:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766167.563443:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114449536:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775766167.563458:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114449664:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775766167.563467:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766167.563474:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766167/real 1775766167]  req@ffff9f14a04d2a00 x1862022114449664/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766192 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766167.563481:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d2a00 x1862022114449664/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766192 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766167.563487:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766167.563488:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766167.563491:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114449664:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775766167.563495:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114449792:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775766167.563505:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766167.563519:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766167/real 1775766167]  req@ffff9f14a04d3100 x1862022114449792/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766192 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766167.563528:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3100 x1862022114449792/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766192 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766167.563533:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766167.563546:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766167.563548:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114449792:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766167.563579:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114449920:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775766167.563594:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766167.563602:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766167/real 1775766167]  req@ffff9f14a04d0e00 x1862022114449920/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766192 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766167.563610:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0e00 x1862022114449920/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766192 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766167.563616:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766167.563618:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766167.563620:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114449920:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766167.563625:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114450048:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775766167.563636:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766167.563642:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766167/real 1775766167]  req@ffff9f14a04d2300 x1862022114450048/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766192 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766167.563649:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d2300 x1862022114450048/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766192 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766167.563655:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766167.563657:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766167.563658:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114450048:192.168.202.110@tcp:8:kworker.0
00010000:00010000:3.0:1775766167.818908:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766168.586645:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766168.842763:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766169.610655:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766169.866743:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766170.634678:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766170.890682:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775766171.658658:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775766171.914709:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775766172.682682:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775766172.682682:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766172.682692:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775766172.682696:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766172.682704:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3550
00000100:00080000:3.0:1775766172.682714:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766172.682720:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775766172.682755:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747100 x1862022114450176/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766172.682781:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766172.682784:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775766172.682786:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766172.682789:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3550
00000100:00080000:3.0:1775766172.682791:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766172.682793:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766172.682801:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766172.682812:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745f80 x1862022114450304/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766172.682820:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1775766172.682822:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114450176:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775766172.682824:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775766172.682828:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766172.682833:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3550
00000100:00080000:3.0:1775766172.682835:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766172.682837:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766172.682840:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766172.682849:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745180 x1862022114450432/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766172.682855:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766172.682858:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775766172.682860:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766172.682862:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3550
00000100:00080000:3.0:1775766172.682864:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766172.682866:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766172.682870:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766172.682877:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745c00 x1862022114450560/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766172.682882:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766172.682884:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775766172.682885:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766172.682888:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3550
00000100:00080000:3.0:1775766172.682890:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000800:00000100:0.0:1775766172.682893:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775766172.682895:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766172.682899:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766172.682905:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747800 x1862022114450688/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766172.682911:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766172.682914:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775766172.682916:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766172.682919:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3550
00000100:00080000:3.0:1775766172.682921:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766172.682924:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766172.682927:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766172.682936:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744000 x1862022114450816/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766172.682942:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1775766172.682943:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766172/real 1775766172]  req@ffff9f1485747100 x1862022114450176/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766197 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766172.682946:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775766172.682949:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766172.682957:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3550
00000100:00080000:3.0:1775766172.682961:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766172.682964:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766172.682967:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775766172.682972:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747100 x1862022114450176/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766197 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766172.682976:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745500 x1862022114450944/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775766172.682993:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766172.682995:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766172.682999:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775766172.683001:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:3.0:1775766172.683002:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775766172.683011:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114450176:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775766172.683011:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3550
00000100:00080000:3.0:1775766172.683016:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766172.683020:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766172.683024:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766172.683030:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744700 x1862022114451072/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775766172.683040:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114450304:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766172.683069:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766172.683084:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766172/real 1775766172]  req@ffff9f1485745f80 x1862022114450304/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766197 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766172.683096:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745f80 x1862022114450304/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766197 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766172.683116:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766172.683120:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766172.683123:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114450304:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766172.683129:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114450432:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766172.683150:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766172.683160:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766172/real 1775766172]  req@ffff9f1485745180 x1862022114450432/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766197 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766172.683170:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745180 x1862022114450432/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766197 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766172.683178:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766172.683181:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766172.683196:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114450432:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766172.683203:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114450560:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766172.683224:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766172.683234:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766172/real 1775766172]  req@ffff9f1485745c00 x1862022114450560/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766197 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766172.683244:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745c00 x1862022114450560/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766197 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766172.683253:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766172.683256:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766172.683259:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114450560:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775766172.683265:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114450688:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766172.683284:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766172.683294:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766172/real 1775766172]  req@ffff9f1485747800 x1862022114450688/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766197 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766172.683340:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747800 x1862022114450688/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766197 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766172.683349:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766172.683352:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766172.683356:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114450688:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775766172.683362:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114450816:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766172.683385:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766172.683397:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766172/real 1775766172]  req@ffff9f1485744000 x1862022114450816/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766197 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766172.683407:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744000 x1862022114450816/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766197 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766172.683413:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766172.683415:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766172.683417:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114450816:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766172.683421:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114450944:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766172.683435:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766172.683442:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766172/real 1775766172]  req@ffff9f1485745500 x1862022114450944/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766197 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766172.683454:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745500 x1862022114450944/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766197 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766172.683459:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766172.683460:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766172.683462:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114450944:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766172.683476:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114451072:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766172.683486:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766172.683492:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766172/real 1775766172]  req@ffff9f1485744700 x1862022114451072/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766197 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766172.683500:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744700 x1862022114451072/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766197 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766172.683504:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766172.683506:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766172.683508:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114451072:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775766172.938735:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775766173.706648:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775766173.962692:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775766174.730646:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775766174.986703:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775766175.754628:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766176.010670:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775766176.778648:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766177.034756:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1775766177.802662:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766177.802671:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775766177.802677:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766177.802686:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3555
00000100:00080000:3.0:1775766177.802698:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00100000:1.0:1775766177.802707:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775766177.802716:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775766177.802759:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746680 x1862022114451200/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766177.802805:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766177.802809:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775766177.802811:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766177.802815:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3555
00000100:00080000:3.0:1775766177.802819:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766177.802823:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766177.802835:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766177.802854:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747b80 x1862022114451328/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766177.802870:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766177.802875:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775766177.802878:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766177.802885:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3555
00000100:00080000:3.0:1775766177.802892:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766177.802898:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775766177.802905:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114451200:192.168.202.110@tcp:250:kworker.0
00000008:00100000:3.0:1775766177.802911:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766177.802945:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744e00 x1862022114451456/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766177.802956:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766177.802959:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775766177.802961:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766177.802965:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3555
00000100:00080000:3.0:1775766177.802970:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766177.802974:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766177.802980:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766177.802990:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744380 x1862022114451584/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766177.802998:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766177.803001:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775766177.803005:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000800:00000100:0.0:1775766177.803007:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775766177.803011:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3555
00000100:00080000:3.0:1775766177.803015:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766177.803018:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766177.803022:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766177.803033:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746a00 x1862022114451712/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766177.803041:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766177.803044:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775766177.803046:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766177.803050:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3555
00000100:00080000:3.0:1775766177.803054:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1775766177.803059:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766177/real 1775766177]  req@ffff9f1485746680 x1862022114451200/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766202 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766177.803060:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766177.803068:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766177.803085:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746300 x1862022114451840/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775766177.803089:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746680 x1862022114451200/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766202 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766177.803113:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766177.803122:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775766177.803126:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766177.803135:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3555
00000100:00080000:3.0:1775766177.803144:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766177.803148:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766177.803166:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766177.803172:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00080000:0.0:1775766177.803185:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766177.803187:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747480 x1862022114451968/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775766177.803196:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114451200:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775766177.803215:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1775766177.803216:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114451328:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775766177.803225:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775766177.803229:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766177.803236:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3555
00000100:00080000:3.0:1775766177.803242:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766177.803246:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000800:00000100:0.0:1775766177.803251:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000008:00100000:3.0:1775766177.803255:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775766177.803288:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766177/real 1775766177]  req@ffff9f1485747b80 x1862022114451328/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766202 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766177.803302:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747b80 x1862022114451328/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766202 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766177.803309:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745880 x1862022114452096/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775766177.803331:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766177.803339:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766177.803344:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114451328:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766177.803368:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114451456:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766177.803386:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766177.803396:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766177/real 1775766177]  req@ffff9f1485744e00 x1862022114451456/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766202 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766177.803405:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744e00 x1862022114451456/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766202 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766177.803411:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766177.803414:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766177.803416:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114451456:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766177.803422:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114451584:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766177.803434:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766177.803440:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766177/real 1775766177]  req@ffff9f1485744380 x1862022114451584/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766202 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766177.803449:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744380 x1862022114451584/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766202 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766177.803455:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766177.803458:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766177.803460:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114451584:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775766177.803464:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114451712:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766177.803475:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766177.803482:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766177/real 1775766177]  req@ffff9f1485746a00 x1862022114451712/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766202 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766177.803490:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746a00 x1862022114451712/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766202 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766177.803496:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766177.803498:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766177.803500:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114451712:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775766177.803504:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114451840:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766177.803515:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766177.803521:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766177/real 1775766177]  req@ffff9f1485746300 x1862022114451840/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766202 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766177.803531:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746300 x1862022114451840/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766202 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766177.803537:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766177.803539:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766177.803541:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114451840:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766177.803545:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114451968:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766177.803584:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766177.803593:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766177/real 1775766177]  req@ffff9f1485747480 x1862022114451968/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766202 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766177.803602:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747480 x1862022114451968/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766202 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766177.803608:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766177.803611:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766177.803613:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114451968:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766177.803618:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114452096:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766177.803630:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766177.803637:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766177/real 1775766177]  req@ffff9f1485745880 x1862022114452096/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766202 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766177.803645:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745880 x1862022114452096/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766202 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766177.803657:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766177.803659:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766177.803662:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114452096:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775766178.058715:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775766178.826658:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766179.082698:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775766179.850622:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766180.106773:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775766180.874662:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775766181.130748:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775766181.898636:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775766182.154705:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1775766182.922708:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:1.0:1775766182.922709:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775766182.922724:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775766182.922731:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766182.922747:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3560
00000100:00080000:3.0:1775766182.922791:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766182.922801:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775766182.922857:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744a80 x1862022114452224/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766182.922901:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766182.922906:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775766182.922908:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766182.922914:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3560
00000100:00080000:3.0:1775766182.922920:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766182.922926:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766182.922953:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766182.922974:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745880 x1862022114452352/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775766182.922981:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114452224:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775766182.923008:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766182.923021:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775766182.923023:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766182.923030:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3560
00000100:00080000:3.0:1775766182.923036:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766182.923043:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766182.923051:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766182.923071:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747480 x1862022114452480/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1775766182.923088:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775766182.923103:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766182.923106:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775766182.923108:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766182.923116:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3560
00000100:00080000:3.0:1775766182.923123:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766182.923129:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766182.923136:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775766182.923152:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766182/real 1775766182]  req@ffff9f1485744a80 x1862022114452224/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766207 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766182.923157:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746300 x1862022114452608/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766182.923216:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1775766182.923238:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744a80 x1862022114452224/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766207 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766182.923240:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775766182.923246:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766182.923262:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3560
00000100:00080000:0.0:1775766182.923269:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766182.923270:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766182.923281:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:3.0:1775766182.923282:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775766182.923293:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114452224:192.168.202.110@tcp:250:kworker.0
00000008:00100000:3.0:1775766182.923295:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766182.923317:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746a00 x1862022114452736/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766182.923327:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766182.923330:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775766182.923332:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775766182.923339:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114452352:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775766182.923342:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3560
00000100:00080000:3.0:1775766182.923350:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766182.923361:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766182.923371:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766182.923384:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744380 x1862022114452864/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1775766182.923396:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775766182.923405:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766182.923409:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775766182.923411:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766182.923417:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3560
00000100:00080000:3.0:1775766182.923422:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1775766182.923426:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766182/real 1775766182]  req@ffff9f1485745880 x1862022114452352/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766207 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766182.923429:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766182.923444:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775766182.923450:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745880 x1862022114452352/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766207 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766182.923457:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744e00 x1862022114452992/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775766182.923477:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766182.923493:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00080000:3.0:1775766182.923505:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1775766182.923509:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114452352:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775766182.923513:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775766182.923520:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766182.923525:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3560
00000100:00100000:0.0:1775766182.923527:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114452480:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775766182.923533:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766182.923539:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766182.923545:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766182.923559:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747b80 x1862022114453120/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1775766182.923583:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766182.923596:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766182/real 1775766182]  req@ffff9f1485747480 x1862022114452480/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766207 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766182.923634:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747480 x1862022114452480/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766207 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766182.923642:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766182.923646:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766182.923649:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114452480:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766182.923654:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114452608:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766182.923670:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766182.923676:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766182/real 1775766182]  req@ffff9f1485746300 x1862022114452608/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766207 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766182.923684:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746300 x1862022114452608/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766207 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766182.923689:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766182.923691:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766182.923693:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114452608:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775766182.923696:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114452736:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766182.923706:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766182.923712:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766182/real 1775766182]  req@ffff9f1485746a00 x1862022114452736/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766207 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766182.923719:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746a00 x1862022114452736/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766207 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766182.923724:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766182.923726:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766182.923728:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114452736:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775766182.923743:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114452864:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766182.923762:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766182.923779:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766182/real 1775766182]  req@ffff9f1485744380 x1862022114452864/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766207 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766182.923791:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744380 x1862022114452864/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766207 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766182.923814:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766182.923817:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766182.923820:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114452864:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766182.923827:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114452992:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766182.923847:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766182.923858:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766182/real 1775766182]  req@ffff9f1485744e00 x1862022114452992/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766207 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766182.923871:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744e00 x1862022114452992/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766207 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766182.923886:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766182.923888:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766182.923891:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114452992:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766182.923895:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114453120:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766182.923911:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766182.923935:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766182/real 1775766182]  req@ffff9f1485747b80 x1862022114453120/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766207 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766182.923942:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747b80 x1862022114453120/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766207 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766182.923948:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766182.923950:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766182.923952:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114453120:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775766183.178676:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775766183.946658:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766184.202732:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775766184.970646:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766185.226811:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775766185.994781:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766186.250654:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775766187.018640:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766187.274739:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775766188.042637:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775766188.042650:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766188.042655:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775766188.042659:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766188.042667:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3565
00000100:00080000:3.0:1775766188.042676:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766188.042682:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775766188.042713:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746680 x1862022114453248/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766188.042734:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766188.042737:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775766188.042738:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766188.042741:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3565
00000100:00080000:3.0:1775766188.042743:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766188.042746:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766188.042753:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766188.042762:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744700 x1862022114453376/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766188.042770:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766188.042773:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775766188.042775:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775766188.042780:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114453248:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775766188.042786:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3565
00000100:00080000:3.0:1775766188.042789:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766188.042791:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766188.042796:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766188.042805:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745500 x1862022114453504/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766188.042812:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766188.042815:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775766188.042816:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766188.042818:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3565
00000100:00080000:3.0:1775766188.042821:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766188.042823:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766188.042826:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766188.042834:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744000 x1862022114453632/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1775766188.042841:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775766188.042848:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766188.042850:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775766188.042852:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766188.042854:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3565
00000100:00080000:3.0:1775766188.042856:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766188.042859:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766188.042862:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766188.042868:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747800 x1862022114453760/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766188.042874:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766188.042877:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775766188.042878:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766188.042882:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3565
00000100:00080000:3.0:1775766188.042885:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766188.042888:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766188.042891:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775766188.042892:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766188/real 1775766188]  req@ffff9f1485746680 x1862022114453248/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766213 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766188.042902:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745c00 x1862022114453888/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766188.042909:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1775766188.042912:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746680 x1862022114453248/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766213 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766188.042913:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775766188.042916:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766188.042925:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3565
00000100:00080000:3.0:1775766188.042929:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766188.042932:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766188.042932:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766188.042936:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00080000:0.0:1775766188.042940:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766188.042944:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114453248:192.168.202.110@tcp:250:kworker.0
00000100:00100000:3.0:1775766188.042949:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745180 x1862022114454016/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766188.042956:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766188.042958:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775766188.042960:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766188.042962:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3565
00000100:00080000:3.0:1775766188.042965:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1775766188.042969:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114453376:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775766188.042969:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766188.042976:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766188.042986:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745f80 x1862022114454144/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1775766188.042997:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766188.043009:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766188/real 1775766188]  req@ffff9f1485744700 x1862022114453376/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766213 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766188.043017:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744700 x1862022114453376/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766213 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766188.043023:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766188.043025:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766188.043028:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114453376:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766188.043032:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114453504:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766188.043042:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766188.043048:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766188/real 1775766188]  req@ffff9f1485745500 x1862022114453504/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766213 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766188.043055:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745500 x1862022114453504/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766213 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766188.043060:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766188.043062:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766188.043064:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114453504:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766188.043068:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114453632:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766188.043078:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766188.043083:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766188/real 1775766188]  req@ffff9f1485744000 x1862022114453632/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766213 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766188.043091:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744000 x1862022114453632/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766213 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766188.043096:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766188.043098:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766188.043100:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114453632:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775766188.043103:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114453760:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766188.043113:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766188.043118:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766188/real 1775766188]  req@ffff9f1485747800 x1862022114453760/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766213 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766188.043125:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747800 x1862022114453760/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766213 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766188.043130:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766188.043132:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766188.043134:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114453760:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775766188.043137:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114453888:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766188.043146:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766188.043151:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766188/real 1775766188]  req@ffff9f1485745c00 x1862022114453888/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766213 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766188.043158:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745c00 x1862022114453888/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766213 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766188.043167:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766188.043185:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766188.043188:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114453888:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766188.043192:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114454016:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766188.043222:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766188.043230:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766188/real 1775766188]  req@ffff9f1485745180 x1862022114454016/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766213 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766188.043238:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745180 x1862022114454016/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766213 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766188.043243:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766188.043245:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766188.043247:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114454016:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766188.043260:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114454144:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766188.043270:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766188.043276:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766188/real 1775766188]  req@ffff9f1485745f80 x1862022114454144/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766213 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766188.043283:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745f80 x1862022114454144/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766213 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766188.043288:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766188.043290:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766188.043293:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114454144:192.168.202.110@tcp:8:kworker.0
00010000:00010000:3.0:1775766188.298715:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775766189.066626:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775766189.322708:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775766190.090664:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766190.346676:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775766191.114641:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766191.370712:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775766192.138718:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766192.394795:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1775766193.162655:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766193.162664:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1775766193.162668:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00100000:3.0:1775766193.162675:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:2.0:1775766193.162678:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3570
00000100:00080000:2.0:1775766193.162691:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766193.162698:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775766193.162731:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3480 x1862022114454272/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766193.162760:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766193.162763:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775766193.162765:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766193.162769:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3570
00000100:00080000:2.0:1775766193.162773:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766193.162776:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766193.162786:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766193.162798:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114454272:192.168.202.110@tcp:250:kworker.0
00000100:00100000:2.0:1775766193.162803:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1c00 x1862022114454400/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766193.162816:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766193.162819:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775766193.162821:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766193.162824:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3570
00000100:00080000:2.0:1775766193.162826:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766193.162829:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766193.162834:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766193.162846:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2300 x1862022114454528/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766193.162854:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766193.162856:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775766193.162858:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766193.162861:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3570
00000100:00080000:2.0:1775766193.162865:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000800:00000100:3.0:1775766193.162867:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:2.0:1775766193.162872:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766193.162877:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775766193.162889:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3100 x1862022114454656/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766193.162898:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766193.162901:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1775766193.162903:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766193.162907:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3570
00000100:00080000:2.0:1775766193.162911:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766193.162917:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775766193.162918:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766193/real 1775766193]  req@ffff9f14852d3480 x1862022114454272/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766218 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:2.0:1775766193.162921:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775766193.162935:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1880 x1862022114454784/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775766193.162943:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3480 x1862022114454272/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766218 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766193.162945:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766193.162952:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775766193.162953:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766193.162957:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3570
00000100:00080000:2.0:1775766193.162960:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766193.162962:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766193.162965:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766193.162972:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0e00 x1862022114454912/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766193.162984:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766193.162986:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775766193.162988:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766193.162990:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3570
00000100:00080000:2.0:1775766193.162992:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766193.162994:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766193.162997:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766193.163002:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3b80 x1862022114455040/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766193.163008:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766193.163010:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775766193.163011:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766193.163013:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3570
00000100:00080000:2.0:1775766193.163015:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766193.163018:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766193.163021:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775766193.163027:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0000 x1862022114455168/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766193.163072:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766193.163078:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766193.163298:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114454272:192.168.202.110@tcp:250:kworker.0
00000100:00100000:3.0:1775766193.163324:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114454400:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775766193.163406:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766193.163420:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766193/real 1775766193]  req@ffff9f14852d1c00 x1862022114454400/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766218 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766193.163433:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1c00 x1862022114454400/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766218 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766193.163441:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766193.163444:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766193.163447:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114454400:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766193.163453:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114454528:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775766193.163467:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766193.163474:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766193/real 1775766193]  req@ffff9f14852d2300 x1862022114454528/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766218 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766193.163484:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2300 x1862022114454528/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766218 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766193.163500:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766193.163503:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766193.163505:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114454528:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766193.163510:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114454656:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775766193.163535:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766193.163543:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766193/real 1775766193]  req@ffff9f14852d3100 x1862022114454656/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766218 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766193.163552:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3100 x1862022114454656/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766218 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766193.163580:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766193.163584:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766193.163588:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114454656:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775766193.163594:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114454784:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775766193.163612:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766193.163621:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766193/real 1775766193]  req@ffff9f14852d1880 x1862022114454784/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766218 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766193.163631:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1880 x1862022114454784/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766218 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766193.163637:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766193.163639:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766193.163641:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114454784:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775766193.163645:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114454912:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775766193.163656:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766193.163662:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766193/real 1775766193]  req@ffff9f14852d0e00 x1862022114454912/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766218 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766193.163671:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0e00 x1862022114454912/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766218 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766193.163677:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766193.163679:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766193.163681:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114454912:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766193.163687:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114455040:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775766193.163708:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766193.163721:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766193/real 1775766193]  req@ffff9f14852d3b80 x1862022114455040/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766218 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766193.163748:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3b80 x1862022114455040/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766218 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766193.163767:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766193.163772:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766193.163775:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114455040:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766193.163780:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114455168:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775766193.163799:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766193.163808:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766193/real 1775766193]  req@ffff9f14852d0000 x1862022114455168/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766218 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766193.163817:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0000 x1862022114455168/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766218 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766193.163823:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766193.163826:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766193.163828:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114455168:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775766193.418669:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775766194.186680:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766194.442718:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775766195.210648:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775766195.466761:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775766196.234631:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775766196.490725:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775766197.258661:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766197.514737:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775766198.282638:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:2.0:1775766198.282641:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766198.282649:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1775766198.282654:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766198.282664:0:8:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3575
00000100:00080000:2.0:1775766198.282676:0:8:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766198.282684:0:8:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775766198.282720:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0380 x1862022114455296/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766198.282743:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766198.282746:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775766198.282747:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766198.282750:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3575
00000100:00080000:2.0:1775766198.282754:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766198.282758:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766198.282766:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766198.282771:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114455296:192.168.202.110@tcp:250:kworker.0
00000100:00100000:2.0:1775766198.282777:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3800 x1862022114455424/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766198.282785:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766198.282787:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775766198.282788:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766198.282791:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3575
00000100:00080000:2.0:1775766198.282793:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766198.282795:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766198.282798:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766198.282804:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0700 x1862022114455552/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766198.282815:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766198.282817:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775766198.282818:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766198.282821:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3575
00000100:00080000:2.0:1775766198.282823:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766198.282826:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766198.282829:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775766198.282836:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2d80 x1862022114455680/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766198.282841:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766198.282844:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1775766198.282845:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766198.282847:0:8:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3575
00000100:00080000:2.0:1775766198.282850:0:8:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766198.282852:0:8:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766198.282856:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775766198.282863:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1180 x1862022114455808/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766198.282870:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:3.0:1775766198.282871:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114455424:192.168.202.110@tcp:38:kworker.0
00000100:00080000:2.0:1775766198.282874:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775766198.282879:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766198.282882:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3575
00000100:00080000:2.0:1775766198.282886:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766198.282890:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766198.282893:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766198.282902:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2680 x1862022114455936/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775766198.282905:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114455552:192.168.202.110@tcp:38:kworker.0
00000100:00080000:2.0:1775766198.282916:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766198.282919:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775766198.282920:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766198.282925:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3575
00000100:00080000:2.0:1775766198.282928:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766198.282932:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775766198.282932:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114455680:192.168.202.110@tcp:8:kworker.0
00000008:00100000:2.0:1775766198.282939:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766198.282952:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2a00 x1862022114456064/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775766198.282960:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114455808:192.168.202.110@tcp:8:kworker.0
00000100:00080000:2.0:1775766198.282963:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766198.282968:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775766198.282970:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766198.282975:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3575
00000100:00100000:3.0:1775766198.282978:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114455936:192.168.202.110@tcp:38:kworker.0
00000100:00080000:2.0:1775766198.282983:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766198.282991:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766198.282998:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766198.283002:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114456064:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766198.283013:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1f80 x1862022114456192/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775766198.283055:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114456192:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775766198.538807:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775766199.306667:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766199.562759:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775766200.330630:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766200.586780:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775766201.354651:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000400:00000100:1.0:1775766201.354829:0:2390:0:(lib-socket.c:454:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.202.110:988
00000400:00000100:1.0:1775766201.354874:0:2390:0:(acceptor.c:111:lnet_connect_console_error()) Connection to 192.168.202.110@tcp at host 192.168.202.110 was unreachable: the network or that node may be down, or Lustre may be misconfigured.
00000800:00000100:1.0:1775766201.354895:0:2390:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:1.0:1775766201.354969:0:2390:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:1.0:1775766201.354982:0:2390:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:1.0:1775766201.354992:0:2390:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:1.0:1775766201.355002:0:2390:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:1.0:1775766201.355012:0:2390:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:1.0:1775766201.355021:0:2390:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:1.0:1775766201.355032:0:2390:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000100:00100000:3.0:1775766201.355160:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766198/real 1775766201]  req@ffff9f14852d1f80 x1862022114456192/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766223 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766201.355194:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1f80 x1862022114456192/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766223 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766201.355210:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766201.355216:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766201.355222:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114456192:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775766201.355232:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766198/real 1775766201]  req@ffff9f14852d2a00 x1862022114456064/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766223 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766201.355259:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2a00 x1862022114456064/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766223 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766201.355271:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766201.355276:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766201.355280:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114456064:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766201.355289:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766198/real 1775766201]  req@ffff9f14852d1180 x1862022114455808/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766223 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766201.355305:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1180 x1862022114455808/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766223 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766201.355314:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766201.355319:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766201.355324:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114455808:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775766201.355332:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766198/real 1775766201]  req@ffff9f14852d2680 x1862022114455936/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766223 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766201.355348:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2680 x1862022114455936/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766223 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766201.355358:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766201.355363:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766201.355368:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114455936:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766201.355377:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766198/real 1775766201]  req@ffff9f14852d3800 x1862022114455424/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766223 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766201.355403:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3800 x1862022114455424/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766223 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766201.355416:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766201.355421:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766201.355426:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114455424:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766201.355436:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766198/real 1775766201]  req@ffff9f14852d0700 x1862022114455552/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766223 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766201.355452:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0700 x1862022114455552/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766223 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766201.355463:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766201.355468:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766201.355473:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114455552:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766201.355484:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766198/real 1775766201]  req@ffff9f14852d2d80 x1862022114455680/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766223 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766201.355502:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2d80 x1862022114455680/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766223 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766201.355511:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766201.355515:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766201.355518:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114455680:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775766201.355527:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766198/real 1775766201]  req@ffff9f14852d0380 x1862022114455296/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766223 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766201.355543:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0380 x1862022114455296/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766223 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766201.355935:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766201.356059:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766201.356065:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114455296:192.168.202.110@tcp:250:kworker.0
00010000:00010000:2.0:1775766201.610752:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775766202.378627:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775766202.634796:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775766203.402649:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:0.0:1775766203.402693:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766203.402716:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1775766203.402720:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766203.402728:0:8:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3580
00000100:00080000:0.0:1775766203.402738:0:8:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766203.402744:0:8:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775766203.402775:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d1180 x1862022114456320/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775766203.402799:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766203.402802:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775766203.402803:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766203.402806:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3580
00000100:00080000:0.0:1775766203.402809:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766203.402811:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775766203.402819:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775766203.402829:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3800 x1862022114456448/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775766203.402838:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766203.402841:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:0.0:1775766203.402843:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:3.0:1775766203.402848:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114456320:192.168.202.110@tcp:250:kworker.0
00000100:00080000:0.0:1775766203.402851:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3580
00000100:00080000:0.0:1775766203.402874:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766203.402879:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775766203.402884:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775766203.402895:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d1500 x1862022114456576/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775766203.402904:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766203.402908:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775766203.402910:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766203.402914:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3580
00000100:00080000:0.0:1775766203.402918:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766203.402922:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775766203.402927:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775766203.402941:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0000 x1862022114456704/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:3.0:1775766203.402946:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:0.0:1775766203.402957:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766203.402961:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1775766203.402963:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766203.402968:0:8:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3580
00000100:00080000:0.0:1775766203.402972:0:8:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766203.402975:0:8:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775766203.402981:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775766203.402993:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3480 x1862022114456832/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775766203.403000:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766203/real 1775766203]  req@ffff9f14a04d1180 x1862022114456320/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766228 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766203.403007:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766203.403012:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775766203.403014:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766203.403019:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3580
00000100:00080000:0.0:1775766203.403051:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766203.403059:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775766203.403065:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766203.403070:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1180 x1862022114456320/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766228 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766203.403080:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d2300 x1862022114456960/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766203.403098:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766203.403100:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766203.403106:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775766203.403109:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:0.0:1775766203.403110:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:3.0:1775766203.403117:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114456320:192.168.202.110@tcp:250:kworker.0
00000100:00080000:0.0:1775766203.403118:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3580
00000100:00080000:0.0:1775766203.403125:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766203.403131:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775766203.403137:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775766203.403149:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0e00 x1862022114457088/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775766203.403158:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114456448:192.168.202.110@tcp:38:kworker.0
00000100:00080000:0.0:1775766203.403164:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766203.403168:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775766203.403170:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766203.403175:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3580
00000100:00080000:0.0:1775766203.403179:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766203.403184:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775766203.403190:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:3.0:1775766203.403199:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766203.403204:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3100 x1862022114457216/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775766203.403219:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766203/real 1775766203]  req@ffff9f14a04d3800 x1862022114456448/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766228 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766203.403237:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3800 x1862022114456448/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766228 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766203.403245:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766203.403249:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766203.403253:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114456448:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766203.403262:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114456576:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775766203.403285:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766203.403297:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766203/real 1775766203]  req@ffff9f14a04d1500 x1862022114456576/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766228 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766203.403310:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1500 x1862022114456576/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766228 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766203.403317:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766203.403320:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766203.403324:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114456576:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766203.403330:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114456704:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775766203.403352:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766203.403364:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766203/real 1775766203]  req@ffff9f14a04d0000 x1862022114456704/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766228 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766203.403376:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0000 x1862022114456704/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766228 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766203.403383:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766203.403386:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766203.403389:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114456704:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775766203.403395:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114456832:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775766203.403413:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766203.403424:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766203/real 1775766203]  req@ffff9f14a04d3480 x1862022114456832/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766228 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766203.403436:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3480 x1862022114456832/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766228 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766203.403444:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766203.403447:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766203.403451:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114456832:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775766203.403457:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114456960:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775766203.403479:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766203.403906:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766203/real 1775766203]  req@ffff9f14a04d2300 x1862022114456960/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766228 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766203.403922:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d2300 x1862022114456960/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766228 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766203.403931:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766203.403936:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766203.403939:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114456960:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766203.403965:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114457088:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775766203.403989:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766203.403997:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766203/real 1775766203]  req@ffff9f14a04d0e00 x1862022114457088/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766228 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766203.404009:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0e00 x1862022114457088/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766228 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766203.404037:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766203.404041:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766203.404044:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114457088:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766203.404050:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114457216:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775766203.404072:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766203.404083:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766203/real 1775766203]  req@ffff9f14a04d3100 x1862022114457216/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766228 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766203.404095:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3100 x1862022114457216/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766228 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766203.404101:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766203.404103:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766203.404105:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114457216:192.168.202.110@tcp:8:kworker.0
00010000:00010000:3.0:1775766203.658724:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775766204.426639:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766204.682761:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775766205.450677:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766205.706749:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775766206.474660:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766206.730810:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775766207.498682:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766207.754823:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775766208.522752:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:2.0:1775766208.522777:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766208.522788:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1775766208.522795:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766208.522810:0:8:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3586
00000100:00080000:2.0:1775766208.522828:0:8:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766208.522838:0:8:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775766208.522901:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1500 x1862022114457344/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766208.522946:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766208.522951:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775766208.522953:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766208.522959:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3586
00000100:00080000:2.0:1775766208.522962:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766208.522967:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766208.523004:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766208.523022:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0a80 x1862022114457472/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766208.523046:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766208.523054:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00100000:3.0:1775766208.523057:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114457344:192.168.202.110@tcp:250:kworker.0
00000100:00080000:2.0:1775766208.523058:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766208.523069:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3586
00000100:00080000:2.0:1775766208.523080:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766208.523086:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766208.523092:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766208.523121:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0380 x1862022114457600/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766208.523133:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766208.523137:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775766208.523139:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766208.523143:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3586
00000100:00080000:2.0:1775766208.523146:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766208.523150:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766208.523157:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775766208.523172:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2d80 x1862022114457728/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766208.523186:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766208.523190:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1775766208.523191:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766208.523195:0:8:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3586
00000100:00080000:2.0:1775766208.523201:0:8:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766208.523206:0:8:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766208.523213:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:3.0:1775766208.523222:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766208.523227:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0700 x1862022114457856/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766208.523239:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766208.523254:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775766208.523257:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766208.523265:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3586
00000100:00080000:2.0:1775766208.523272:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766208.523278:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766208.523284:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766208.523306:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3800 x1862022114457984/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775766208.523313:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766208/real 1775766208]  req@ffff9f14852d1500 x1862022114457344/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766233 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766208.523397:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766208.523403:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775766208.523407:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:3.0:1775766208.523411:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1500 x1862022114457344/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766233 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766208.523423:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3586
00000100:00080000:2.0:1775766208.523432:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766208.523453:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766208.523474:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00080000:3.0:1775766208.523475:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766208.523489:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766208.523495:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114457344:192.168.202.110@tcp:250:kworker.0
00000100:00100000:2.0:1775766208.523507:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2680 x1862022114458112/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766208.523533:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:3.0:1775766208.523537:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114457472:192.168.202.110@tcp:38:kworker.0
00000100:00080000:2.0:1775766208.523548:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000800:00000100:3.0:1775766208.523651:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:2.0:1775766208.523653:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766208.523662:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3586
00000100:00080000:2.0:1775766208.523670:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766208.523676:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766208.523685:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766208.523692:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766208/real 1775766208]  req@ffff9f14852d0a80 x1862022114457472/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766233 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766208.523708:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1180 x1862022114458240/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775766208.523761:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0a80 x1862022114457472/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766233 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766208.523781:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766208.523787:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766208.523791:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114457472:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766208.523800:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114457600:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775766208.523829:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766208.523842:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766208/real 1775766208]  req@ffff9f14852d0380 x1862022114457600/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766233 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766208.523854:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0380 x1862022114457600/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766233 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766208.523862:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766208.523865:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766208.523869:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114457600:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766208.523875:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114457728:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775766208.523892:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766208.523901:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766208/real 1775766208]  req@ffff9f14852d2d80 x1862022114457728/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766233 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766208.523913:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2d80 x1862022114457728/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766233 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766208.523921:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766208.523924:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766208.523927:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114457728:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775766208.523933:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114457856:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775766208.523948:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766208.523956:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766208/real 1775766208]  req@ffff9f14852d0700 x1862022114457856/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766233 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766208.523968:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0700 x1862022114457856/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766233 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766208.524004:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766208.524009:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766208.524015:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114457856:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775766208.524025:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114457984:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775766208.524060:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766208.524078:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766208/real 1775766208]  req@ffff9f14852d3800 x1862022114457984/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766233 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766208.524124:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3800 x1862022114457984/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766233 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766208.524134:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766208.524138:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766208.524142:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114457984:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766208.524170:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114458112:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775766208.524196:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766208.524234:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766208/real 1775766208]  req@ffff9f14852d2680 x1862022114458112/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766233 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766208.524246:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2680 x1862022114458112/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766233 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766208.524254:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766208.524258:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766208.524260:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114458112:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766208.524266:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114458240:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775766208.524282:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766208.524291:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766208/real 1775766208]  req@ffff9f14852d1180 x1862022114458240/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766233 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766208.524302:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1180 x1862022114458240/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766233 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766208.524316:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766208.524319:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766208.524322:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114458240:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775766208.778869:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775766209.546698:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775766209.802875:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775766210.570718:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775766210.826778:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775766211.594743:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766211.850811:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775766212.618654:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766212.874818:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1775766213.642766:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:3.0:1775766213.642776:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:1.0:1775766213.642789:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1775766213.642797:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766213.642817:0:8:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3591
00000100:00080000:1.0:1775766213.642836:0:8:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766213.642848:0:8:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775766213.642910:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff100 x1862022114458368/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766213.643010:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766213.643033:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775766213.643036:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766213.643046:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3591
00000100:00080000:1.0:1775766213.643054:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766213.643061:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775766213.643070:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114458368:192.168.202.110@tcp:250:kworker.0
00000008:00100000:1.0:1775766213.643078:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775766213.643104:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fea00 x1862022114458496/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766213.643123:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766213.643129:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775766213.643133:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766213.643156:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3591
00000100:00080000:1.0:1775766213.643163:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000800:00000100:3.0:1775766213.643171:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:1.0:1775766213.643178:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766213.643188:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775766213.643211:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fc380 x1862022114458624/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775766213.643219:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766213/real 1775766213]  req@ffff9f14ab4ff100 x1862022114458368/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766238 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766213.643255:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766213.643261:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775766213.643264:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:3.0:1775766213.643270:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff100 x1862022114458368/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766238 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766213.643278:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3591
00000100:00080000:1.0:1775766213.643289:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766213.643301:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775766213.643302:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00080000:3.0:1775766213.643316:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000008:00100000:1.0:1775766213.643321:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766213.643323:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114458368:192.168.202.110@tcp:250:kworker.0
00000100:00100000:1.0:1775766213.643354:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fe300 x1862022114458752/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775766213.643358:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114458496:192.168.202.110@tcp:38:kworker.0
00000100:00080000:1.0:1775766213.643388:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000800:00000100:3.0:1775766213.643395:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:1.0:1775766213.643396:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1775766213.643401:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:3.0:1775766213.643410:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766213/real 1775766213]  req@ffff9f14ab4fea00 x1862022114458496/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766238 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766213.643435:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fea00 x1862022114458496/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766238 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766213.643436:0:8:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3591
00000100:00080000:1.0:1775766213.643454:0:8:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766213.643469:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775766213.643475:0:8:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00080000:3.0:1775766213.643479:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766213.643491:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114458496:192.168.202.110@tcp:38:kworker.0
00000008:00100000:1.0:1775766213.643492:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766213.643509:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114458624:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775766213.643523:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fd880 x1862022114458880/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:3.0:1775766213.643541:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:1.0:1775766213.643542:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766213.643551:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775766213.643554:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766213.643564:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3591
00000100:00080000:1.0:1775766213.643573:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766213.643582:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766213.643592:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766213.643610:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766213/real 1775766213]  req@ffff9f14ab4fc380 x1862022114458624/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766238 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766213.643629:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fc380 x1862022114458624/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766238 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766213.643639:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766213.643641:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766213.643644:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114458624:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775766213.643661:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff100 x1862022114459008/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775766213.643662:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114458752:192.168.202.110@tcp:8:kworker.0
00000100:00080000:1.0:1775766213.643692:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000800:00000100:3.0:1775766213.643702:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:1.0:1775766213.643711:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775766213.643716:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766213.643728:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3591
00000100:00100000:3.0:1775766213.643729:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766213/real 1775766213]  req@ffff9f14ab4fe300 x1862022114458752/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766238 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766213.643740:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766213.643755:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775766213.643763:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fe300 x1862022114458752/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766238 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:1.0:1775766213.643769:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00080000:3.0:1775766213.643789:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766213.643795:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766213.643802:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114458752:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775766213.643811:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fce00 x1862022114459136/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775766213.643819:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114458880:192.168.202.110@tcp:8:kworker.0
00000100:00080000:1.0:1775766213.643849:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766213.643854:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775766213.643865:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000800:00000100:3.0:1775766213.643865:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:1.0:1775766213.643881:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3591
00000100:00080000:1.0:1775766213.643890:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:3.0:1775766213.643890:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766213/real 1775766213]  req@ffff9f14ab4fd880 x1862022114458880/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766238 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766213.643902:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766213.643919:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766213.643943:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fd880 x1862022114458880/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766238 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766213.643957:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766213.643975:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766213.643985:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114458880:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775766213.643986:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fca80 x1862022114459264/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775766213.644008:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114459008:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775766213.644041:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766213.644062:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766213/real 1775766213]  req@ffff9f14ab4ff100 x1862022114459008/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766238 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766213.644091:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff100 x1862022114459008/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766238 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766213.644100:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766213.644103:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766213.644107:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114459008:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766213.644113:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114459136:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775766213.644130:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766213.644138:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766213/real 1775766213]  req@ffff9f14ab4fce00 x1862022114459136/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766238 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766213.644158:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fce00 x1862022114459136/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766238 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766213.644163:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766213.644166:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766213.644168:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114459136:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766213.644177:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114459264:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775766213.644187:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766213.644193:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766213/real 1775766213]  req@ffff9f14ab4fca80 x1862022114459264/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766238 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766213.644201:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fca80 x1862022114459264/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766238 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766213.644206:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766213.644208:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766213.644210:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114459264:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775766213.898792:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775766214.666652:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766214.922749:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775766215.690741:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766215.946767:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775766216.714639:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775766216.970762:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775766217.738649:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775766217.994759:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1775766218.762672:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:3.0:1775766218.762676:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:1.0:1775766218.762688:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1775766218.762693:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766218.762706:0:8:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3596
00000100:00080000:1.0:1775766218.762721:0:8:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766218.762731:0:8:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775766218.762776:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fed80 x1862022114459392/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766218.762806:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766218.762812:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775766218.762814:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766218.762818:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3596
00000100:00080000:1.0:1775766218.762822:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766218.762827:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766218.762838:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766218.762848:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114459392:192.168.202.110@tcp:250:kworker.0
00000100:00100000:1.0:1775766218.762856:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdc00 x1862022114459520/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766218.762868:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766218.762872:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775766218.762874:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766218.762908:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3596
00000100:00080000:1.0:1775766218.762933:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766218.762940:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766218.762949:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000800:00000100:3.0:1775766218.762966:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775766218.762972:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdf80 x1862022114459648/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766218.762986:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766218.762991:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775766218.762994:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766218.762998:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3596
00000100:00080000:1.0:1775766218.763003:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766218.763008:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766218.763015:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766218.763024:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766218/real 1775766218]  req@ffff9f14ab4fed80 x1862022114459392/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766243 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775766218.763032:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ffb80 x1862022114459776/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766218.763062:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766218.763066:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1775766218.763070:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:3.0:1775766218.763072:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fed80 x1862022114459392/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766243 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766218.763079:0:8:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3596
00000100:00080000:1.0:1775766218.763086:0:8:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766218.763096:0:8:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00080000:3.0:1775766218.763099:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000008:00100000:1.0:1775766218.763101:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00080000:3.0:1775766218.763111:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775766218.763119:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff800 x1862022114459904/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775766218.763120:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114459392:192.168.202.110@tcp:250:kworker.0
00000100:00080000:1.0:1775766218.763142:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766218.763146:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775766218.763148:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766218.763170:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3596
00000100:00080000:1.0:1775766218.763174:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:3.0:1775766218.763178:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114459520:192.168.202.110@tcp:38:kworker.0
00000100:00080000:1.0:1775766218.763182:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766218.763192:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775766218.763207:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fed80 x1862022114460032/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766218.763218:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000800:00000100:3.0:1775766218.763219:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:1.0:1775766218.763226:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775766218.763230:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766218.763236:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3596
00000100:00080000:1.0:1775766218.763241:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766218.763249:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775766218.763250:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766218/real 1775766218]  req@ffff9f14ab4fdc00 x1862022114459520/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766243 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:1.0:1775766218.763253:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775766218.763268:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fca80 x1862022114460160/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766218.763280:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766218.763283:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775766218.763286:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:3.0:1775766218.763287:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdc00 x1862022114459520/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766243 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766218.763292:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3596
00000100:00080000:1.0:1775766218.763301:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766218.763312:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00080000:3.0:1775766218.763315:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000008:00100000:1.0:1775766218.763320:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00080000:3.0:1775766218.763325:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766218.763333:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114459520:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775766218.763343:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fce00 x1862022114460288/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775766218.763348:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114459648:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775766218.763397:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766218.763412:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766218/real 1775766218]  req@ffff9f14ab4fdf80 x1862022114459648/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766243 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766218.763430:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdf80 x1862022114459648/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766243 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766218.763437:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766218.763440:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766218.763443:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114459648:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766218.763448:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ffb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114459776:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775766218.763465:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766218.763472:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766218/real 1775766218]  req@ffff9f14ab4ffb80 x1862022114459776/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766243 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766218.763483:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ffb80 x1862022114459776/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766243 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766218.763490:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766218.763492:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766218.763495:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ffb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114459776:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775766218.763499:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114459904:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775766218.763512:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766218.763519:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766218/real 1775766218]  req@ffff9f14ab4ff800 x1862022114459904/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766243 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766218.763528:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff800 x1862022114459904/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766243 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766218.763540:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766218.763543:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766218.763545:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114459904:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775766218.763562:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114460032:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775766218.763575:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766218.763582:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766218/real 1775766218]  req@ffff9f14ab4fed80 x1862022114460032/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766243 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766218.763618:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fed80 x1862022114460032/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766243 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766218.763626:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766218.763629:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766218.763632:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114460032:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766218.763638:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114460160:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775766218.763654:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766218.763663:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766218/real 1775766218]  req@ffff9f14ab4fca80 x1862022114460160/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766243 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766218.763673:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fca80 x1862022114460160/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766243 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766218.763680:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766218.763682:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766218.763686:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114460160:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766218.763690:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114460288:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775766218.763703:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766218.763710:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766218/real 1775766218]  req@ffff9f14ab4fce00 x1862022114460288/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766243 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766218.763720:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fce00 x1862022114460288/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766243 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766218.763727:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766218.763729:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766218.763731:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114460288:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775766219.018793:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775766219.786655:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766220.042840:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775766220.810703:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766221.066940:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775766221.834672:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766222.090925:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775766222.858626:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766223.114730:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775766223.882646:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:0.0:1775766223.882672:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766223.882681:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1775766223.882685:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766223.882696:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3601
00000100:00080000:0.0:1775766223.882723:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766223.882732:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775766223.882773:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d2a00 x1862022114460416/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775766223.882803:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766223.882807:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775766223.882809:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766223.882815:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3601
00000100:00080000:0.0:1775766223.882820:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766223.882837:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775766223.882846:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114460416:192.168.202.110@tcp:250:kworker.0
00000008:00100000:0.0:1775766223.882850:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775766223.882883:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0380 x1862022114460544/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775766223.882897:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766223.882901:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:0.0:1775766223.882903:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766223.882909:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3601
00000100:00080000:0.0:1775766223.882916:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000800:00000100:3.0:1775766223.882918:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:0.0:1775766223.882926:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775766223.882935:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775766223.882952:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d1f80 x1862022114460672/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775766223.882959:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766223/real 1775766223]  req@ffff9f14a04d2a00 x1862022114460416/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766248 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766223.882974:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766223.882978:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775766223.882981:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:3.0:1775766223.882982:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d2a00 x1862022114460416/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766248 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766223.882993:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3601
00000100:00080000:3.0:1775766223.882994:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766223.882999:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766223.883001:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:0.0:1775766223.883007:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775766223.883008:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114460416:192.168.202.110@tcp:250:kworker.0
00000008:00100000:0.0:1775766223.883019:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766223.883032:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114460544:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766223.883039:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3b80 x1862022114460800/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775766223.883052:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000800:00000100:3.0:1775766223.883052:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:0.0:1775766223.883058:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1775766223.883061:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:3.0:1775766223.883063:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766223/real 1775766223]  req@ffff9f14a04d0380 x1862022114460544/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766248 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766223.883071:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3601
00000100:00080000:0.0:1775766223.883079:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00100000:3.0:1775766223.883080:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0380 x1862022114460544/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766248 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766223.883090:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00080000:3.0:1775766223.883091:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766223.883098:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000008:00100000:0.0:1775766223.883101:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766223.883101:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114460544:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766223.883110:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114460672:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766223.883126:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0a80 x1862022114460928/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:3.0:1775766223.883129:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766223.883138:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766223/real 1775766223]  req@ffff9f14a04d1f80 x1862022114460672/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766248 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766223.883160:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1f80 x1862022114460672/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766248 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766223.883169:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766223.883169:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766223.883175:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00080000:0.0:1775766223.883180:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00100000:3.0:1775766223.883180:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114460672:192.168.202.110@tcp:38:kworker.0
00000100:00080000:0.0:1775766223.883186:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766223.883194:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3601
00000100:00100000:3.0:1775766223.883196:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114460800:192.168.202.110@tcp:8:kworker.0
00000100:00080000:0.0:1775766223.883203:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766223.883211:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000800:00000100:3.0:1775766223.883214:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000008:00100000:0.0:1775766223.883223:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766223.883227:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766223/real 1775766223]  req@ffff9f14a04d3b80 x1862022114460800/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766248 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766223.883245:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d1f80 x1862022114461056/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775766223.883246:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3b80 x1862022114460800/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766248 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766223.883261:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766223.883273:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:0.0:1775766223.883278:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766223.883280:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766223.883286:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3601
00000100:00080000:3.0:1775766223.883288:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00080000:0.0:1775766223.883295:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:3.0:1775766223.883295:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114460800:192.168.202.110@tcp:8:kworker.0
00000100:00080000:0.0:1775766223.883305:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775766223.883306:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114460928:192.168.202.110@tcp:8:kworker.0
00000008:00100000:0.0:1775766223.883318:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775766223.883334:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0380 x1862022114461184/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:3.0:1775766223.883336:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766223.883349:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766223/real 1775766223]  req@ffff9f14a04d0a80 x1862022114460928/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766248 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766223.883351:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766223.883364:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775766223.883367:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:3.0:1775766223.883368:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0a80 x1862022114460928/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766248 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766223.883379:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3601
00000100:00080000:3.0:1775766223.883380:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766223.883388:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766223.883392:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00080000:3.0:1775766223.883395:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766223.883401:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114460928:192.168.202.110@tcp:8:kworker.0
00000008:00100000:0.0:1775766223.883405:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766223.883414:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114461056:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766223.883423:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d2a00 x1862022114461312/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:3.0:1775766223.883433:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766223.883445:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766223/real 1775766223]  req@ffff9f14a04d1f80 x1862022114461056/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766248 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766223.883455:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1f80 x1862022114461056/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766248 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766223.883462:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766223.883464:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766223.883467:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114461056:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766223.883476:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114461184:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775766223.883488:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766223.883494:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766223/real 1775766223]  req@ffff9f14a04d0380 x1862022114461184/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766248 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766223.883502:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0380 x1862022114461184/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766248 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766223.883512:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766223.883514:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766223.883516:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114461184:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766223.883525:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114461312:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775766223.883536:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766223.883542:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766223/real 1775766223]  req@ffff9f14a04d2a00 x1862022114461312/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766248 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766223.883549:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d2a00 x1862022114461312/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766248 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766223.883573:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766223.883575:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766223.883578:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114461312:192.168.202.110@tcp:8:kworker.0
00010000:00010000:3.0:1775766224.138719:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775766224.906628:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766225.162742:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775766225.930631:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766226.186715:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775766226.954735:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766227.210701:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775766227.978677:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766228.234749:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1775766229.002672:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:3.0:1775766229.002676:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:0.0:1775766229.002681:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1775766229.002685:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766229.002693:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3606
00000100:00080000:0.0:1775766229.002703:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766229.002710:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775766229.002753:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3100 x1862022114461440/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775766229.002790:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766229.002793:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775766229.002794:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766229.002797:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3606
00000100:00080000:0.0:1775766229.002800:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766229.002803:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775766229.002812:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775766229.002824:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0e00 x1862022114461568/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:1.0:1775766229.002830:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114461440:192.168.202.110@tcp:250:kworker.0
00000100:00080000:0.0:1775766229.002834:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766229.002839:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:0.0:1775766229.002841:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766229.002845:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3606
00000100:00080000:0.0:1775766229.002849:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766229.002852:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775766229.002856:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775766229.002868:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d2300 x1862022114461696/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775766229.002874:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766229.002877:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775766229.002878:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766229.002881:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3606
00000100:00080000:0.0:1775766229.002884:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766229.002887:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000800:00000100:1.0:1775766229.002892:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000008:00100000:0.0:1775766229.002893:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775766229.002906:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3480 x1862022114461824/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775766229.002916:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766229.002919:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1775766229.002922:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766229.002928:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3606
00000100:00100000:1.0:1775766229.002928:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766229/real 1775766229]  req@ffff9f14a04d3100 x1862022114461440/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766254 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766229.002936:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766229.002944:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775766229.002949:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775766229.002957:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3100 x1862022114461440/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766254 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766229.002963:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0000 x1862022114461952/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766229.002979:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766229.002983:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766229.002987:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:0.0:1775766229.002988:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775766229.002992:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:1.0:1775766229.002995:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114461440:192.168.202.110@tcp:250:kworker.0
00000100:00080000:0.0:1775766229.002999:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3606
00000100:00080000:0.0:1775766229.003004:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766229.003008:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775766229.003011:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775766229.003021:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d1500 x1862022114462080/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:1.0:1775766229.003021:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114461568:192.168.202.110@tcp:38:kworker.0
00000100:00080000:0.0:1775766229.003035:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766229.003038:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:0.0:1775766229.003039:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766229.003043:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3606
00000100:00080000:0.0:1775766229.003047:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000800:00000100:1.0:1775766229.003048:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:0.0:1775766229.003052:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775766229.003056:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775766229.003063:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766229/real 1775766229]  req@ffff9f14a04d0e00 x1862022114461568/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766254 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766229.003067:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3800 x1862022114462208/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775766229.003087:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:1.0:1775766229.003090:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0e00 x1862022114461568/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766254 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766229.003091:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775766229.003095:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766229.003103:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3606
00000100:00080000:0.0:1775766229.003106:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766229.003110:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00080000:1.0:1775766229.003114:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000008:00100000:0.0:1775766229.003117:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00080000:1.0:1775766229.003119:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775766229.003125:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114461568:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775766229.003130:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114461696:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766229.003136:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d1180 x1862022114462336/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:1.0:1775766229.003148:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775766229.003161:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766229/real 1775766229]  req@ffff9f14a04d2300 x1862022114461696/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766254 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775766229.003172:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d2300 x1862022114461696/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766254 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766229.003178:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775766229.003180:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775766229.003182:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114461696:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775766229.003188:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114461824:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775766229.003206:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775766229.003242:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766229/real 1775766229]  req@ffff9f14a04d3480 x1862022114461824/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766254 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775766229.003251:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3480 x1862022114461824/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766254 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766229.003257:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775766229.003260:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775766229.003263:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114461824:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775766229.003267:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114461952:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775766229.003279:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775766229.003285:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766229/real 1775766229]  req@ffff9f14a04d0000 x1862022114461952/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766254 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775766229.003292:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0000 x1862022114461952/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766254 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766229.003308:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775766229.003310:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775766229.003312:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114461952:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775766229.003324:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114462080:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775766229.003333:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775766229.003346:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766229/real 1775766229]  req@ffff9f14a04d1500 x1862022114462080/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766254 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775766229.003358:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1500 x1862022114462080/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766254 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766229.003363:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775766229.003365:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775766229.003367:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114462080:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775766229.003370:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114462208:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775766229.003380:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775766229.003385:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766229/real 1775766229]  req@ffff9f14a04d3800 x1862022114462208/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766254 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775766229.003392:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3800 x1862022114462208/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766254 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766229.003397:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775766229.003399:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775766229.003400:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114462208:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775766229.003404:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114462336:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775766229.003413:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775766229.003418:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766229/real 1775766229]  req@ffff9f14a04d1180 x1862022114462336/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766254 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775766229.003425:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1180 x1862022114462336/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766254 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766229.003434:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775766229.003436:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775766229.003437:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114462336:192.168.202.110@tcp:8:kworker.0
00010000:00010000:1.0:1775766229.258733:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775766230.026628:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766230.282792:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775766231.050801:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775766231.306749:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775766232.074683:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766232.330823:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775766233.098675:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766233.354749:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775766234.122668:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:0.0:1775766234.122685:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766234.122692:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1775766234.122696:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766234.122704:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3611
00000100:00080000:0.0:1775766234.122714:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766234.122719:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775766234.122773:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d1180 x1862022114462464/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775766234.122796:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766234.122799:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775766234.122800:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766234.122803:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3611
00000100:00080000:0.0:1775766234.122806:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766234.122808:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775766234.122816:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775766234.122824:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3800 x1862022114462592/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775766234.122837:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766234.122842:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00100000:1.0:1775766234.122843:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114462464:192.168.202.110@tcp:250:kworker.0
00000100:00080000:0.0:1775766234.122844:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766234.122851:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3611
00000100:00080000:0.0:1775766234.122853:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766234.122855:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775766234.122858:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775766234.122867:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d1500 x1862022114462720/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775766234.122872:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766234.122875:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775766234.122877:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766234.122879:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3611
00000100:00080000:0.0:1775766234.122882:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766234.122885:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775766234.122888:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775766234.122893:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0000 x1862022114462848/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775766234.122899:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766234.122901:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1775766234.122902:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766234.122905:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3611
00000800:00000100:1.0:1775766234.122910:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:0.0:1775766234.122911:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766234.122915:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775766234.122918:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775766234.122926:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3480 x1862022114462976/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775766234.122932:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766234.122935:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775766234.122936:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766234.122939:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3611
00000100:00080000:0.0:1775766234.122942:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:1.0:1775766234.122946:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766234/real 1775766234]  req@ffff9f14a04d1180 x1862022114462464/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766259 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766234.122947:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775766234.122953:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775766234.122964:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d2300 x1862022114463104/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:1.0:1775766234.122970:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1180 x1862022114462464/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766259 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766234.122976:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766234.122984:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:0.0:1775766234.122987:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766234.122991:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766234.122992:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3611
00000100:00080000:0.0:1775766234.122997:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766234.123001:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:0.0:1775766234.123004:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775766234.123008:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775766234.123008:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114462464:192.168.202.110@tcp:250:kworker.0
00000100:00100000:0.0:1775766234.123022:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0e00 x1862022114463232/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775766234.123033:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:1.0:1775766234.123035:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114462592:192.168.202.110@tcp:38:kworker.0
00000100:00080000:0.0:1775766234.123036:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775766234.123039:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766234.123043:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3611
00000100:00080000:0.0:1775766234.123046:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766234.123048:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775766234.123052:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:1.0:1775766234.123058:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766234.123062:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3100 x1862022114463360/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:1.0:1775766234.123069:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766234/real 1775766234]  req@ffff9f14a04d3800 x1862022114462592/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766259 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775766234.123082:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3800 x1862022114462592/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766259 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766234.123088:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775766234.123091:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775766234.123094:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114462592:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775766234.123098:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114462720:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775766234.123109:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775766234.123115:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766234/real 1775766234]  req@ffff9f14a04d1500 x1862022114462720/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766259 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775766234.123122:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1500 x1862022114462720/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766259 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766234.123126:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775766234.123128:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775766234.123130:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114462720:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775766234.123133:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114462848:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775766234.123143:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775766234.123148:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766234/real 1775766234]  req@ffff9f14a04d0000 x1862022114462848/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766259 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775766234.123155:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0000 x1862022114462848/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766259 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766234.123160:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775766234.123161:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775766234.123163:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114462848:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775766234.123166:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114462976:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775766234.123174:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775766234.123180:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766234/real 1775766234]  req@ffff9f14a04d3480 x1862022114462976/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766259 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775766234.123187:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3480 x1862022114462976/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766259 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766234.123192:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775766234.123193:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775766234.123195:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114462976:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775766234.123198:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114463104:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775766234.123207:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775766234.123211:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766234/real 1775766234]  req@ffff9f14a04d2300 x1862022114463104/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766259 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775766234.123218:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d2300 x1862022114463104/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766259 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766234.123223:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775766234.123225:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775766234.123227:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114463104:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775766234.123239:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114463232:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775766234.123248:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775766234.123253:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766234/real 1775766234]  req@ffff9f14a04d0e00 x1862022114463232/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766259 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775766234.123261:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0e00 x1862022114463232/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766259 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766234.123266:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775766234.123268:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775766234.123270:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114463232:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775766234.123273:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114463360:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775766234.123281:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775766234.123286:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766234/real 1775766234]  req@ffff9f14a04d3100 x1862022114463360/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766259 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775766234.123293:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3100 x1862022114463360/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766259 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766234.123297:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775766234.123299:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775766234.123301:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114463360:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775766234.378672:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775766235.146706:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766235.402746:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775766236.170644:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775766236.426672:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775766237.194640:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766237.450673:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775766238.218680:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775766238.474749:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1775766239.242637:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766239.242645:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775766239.242648:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766239.242657:0:8:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3616
00000100:00080000:3.0:1775766239.242666:0:8:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766239.242672:0:8:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775766239.242696:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:3.0:1775766239.242722:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747100 x1862022114463488/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766239.242745:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766239.242747:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775766239.242749:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766239.242752:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3616
00000100:00080000:3.0:1775766239.242755:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766239.242758:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766239.242766:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766239.242775:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745f80 x1862022114463616/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766239.242783:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766239.242787:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00100000:1.0:1775766239.242789:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114463488:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775766239.242789:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766239.242796:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3616
00000100:00080000:3.0:1775766239.242799:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766239.242802:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766239.242805:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766239.242813:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745180 x1862022114463744/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766239.242819:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766239.242821:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775766239.242822:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766239.242825:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3616
00000100:00080000:3.0:1775766239.242833:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766239.242835:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766239.242839:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766239.242846:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745c00 x1862022114463872/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:1.0:1775766239.242852:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775766239.242860:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766239.242863:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775766239.242865:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766239.242871:0:8:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3616
00000100:00080000:3.0:1775766239.242874:0:8:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766239.242878:0:8:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766239.242885:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775766239.242890:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766239/real 1775766239]  req@ffff9f1485747100 x1862022114463488/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766264 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775766239.242905:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747100 x1862022114463488/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766264 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766239.242909:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747800 x1862022114464000/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766239.242916:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775766239.242924:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775766239.242939:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114463488:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775766239.242939:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766239.242945:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775766239.242946:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766239.242950:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3616
00000100:00080000:3.0:1775766239.242954:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766239.242957:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775766239.242961:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114463616:192.168.202.110@tcp:38:kworker.0
00000008:00100000:3.0:1775766239.242962:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766239.242976:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744000 x1862022114464128/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:1.0:1775766239.242984:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775766239.242984:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766239.242989:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775766239.242990:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766239.242995:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3616
00000100:00080000:3.0:1775766239.242997:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:1.0:1775766239.242999:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766239/real 1775766239]  req@ffff9f1485745f80 x1862022114463616/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766264 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766239.243002:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766239.243009:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775766239.243019:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745f80 x1862022114463616/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766264 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766239.243021:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745500 x1862022114464256/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766239.243046:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766239.243047:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766239.243050:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775766239.243051:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766239.243053:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3616
00000100:00080000:3.0:1775766239.243055:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766239.243057:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766239.243059:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00080000:1.0:1775766239.243063:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766239.243067:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744700 x1862022114464384/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:1.0:1775766239.243069:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114463616:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775766239.243081:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114463744:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775766239.243096:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775766239.243105:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766239/real 1775766239]  req@ffff9f1485745180 x1862022114463744/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766264 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775766239.243113:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745180 x1862022114463744/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766264 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766239.243119:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775766239.243121:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775766239.243123:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114463744:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775766239.243132:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114463872:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775766239.243144:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775766239.243150:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766239/real 1775766239]  req@ffff9f1485745c00 x1862022114463872/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766264 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775766239.243157:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745c00 x1862022114463872/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766264 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766239.243163:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775766239.243165:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775766239.243166:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114463872:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775766239.243170:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114464000:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775766239.243178:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775766239.243183:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766239/real 1775766239]  req@ffff9f1485747800 x1862022114464000/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766264 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775766239.243190:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747800 x1862022114464000/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766264 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766239.243195:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775766239.243197:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775766239.243199:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114464000:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775766239.243217:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114464128:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775766239.243226:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775766239.243231:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766239/real 1775766239]  req@ffff9f1485744000 x1862022114464128/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766264 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775766239.243239:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744000 x1862022114464128/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766264 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766239.243244:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775766239.243246:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775766239.243248:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114464128:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775766239.243252:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114464256:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775766239.243260:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775766239.243265:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766239/real 1775766239]  req@ffff9f1485745500 x1862022114464256/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766264 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775766239.243272:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745500 x1862022114464256/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766264 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766239.243277:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775766239.243278:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775766239.243280:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114464256:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775766239.243283:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114464384:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775766239.243292:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775766239.243296:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766239/real 1775766239]  req@ffff9f1485744700 x1862022114464384/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766264 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775766239.243303:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744700 x1862022114464384/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766264 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766239.243307:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775766239.243309:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775766239.243311:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114464384:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775766239.498758:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766240.266636:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766240.522785:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766241.290644:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766241.547650:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766242.314638:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766242.570767:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775766243.338709:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775766243.594726:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1775766244.362683:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766244.362694:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775766244.362699:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00100000:1.0:1775766244.362706:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775766244.362711:0:8:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3621
00000100:00080000:3.0:1775766244.362728:0:8:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766244.362736:0:8:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775766244.362771:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746680 x1862022114464512/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766244.362795:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766244.362799:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775766244.362801:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766244.362806:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3621
00000100:00080000:3.0:1775766244.362810:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766244.362814:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766244.362824:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766244.362838:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747b80 x1862022114464640/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:1.0:1775766244.362849:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114464512:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775766244.362857:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766244.362861:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775766244.362862:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766244.362866:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3621
00000100:00080000:3.0:1775766244.362870:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766244.362874:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766244.362879:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766244.362897:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744e00 x1862022114464768/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766244.362904:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766244.362907:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775766244.362908:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766244.362910:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3621
00000100:00080000:3.0:1775766244.362913:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766244.362916:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766244.362919:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766244.362926:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744380 x1862022114464896/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:1.0:1775766244.362936:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775766244.362950:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766244.362953:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775766244.362956:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766244.362963:0:8:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3621
00000100:00080000:3.0:1775766244.362967:0:8:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766244.362971:0:8:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766244.362977:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775766244.362992:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766244/real 1775766244]  req@ffff9f1485746680 x1862022114464512/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766269 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766244.362997:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746a00 x1862022114465024/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766244.363027:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:1.0:1775766244.363032:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746680 x1862022114464512/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766269 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766244.363033:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775766244.363038:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766244.363055:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3621
00000100:00080000:1.0:1775766244.363061:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766244.363061:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766244.363078:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00080000:1.0:1775766244.363079:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775766244.363088:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114464512:192.168.202.110@tcp:250:kworker.0
00000008:00100000:3.0:1775766244.363089:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766244.363111:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746300 x1862022114465152/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766244.363119:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766244.363122:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00100000:1.0:1775766244.363126:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114464640:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775766244.363126:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766244.363145:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3621
00000100:00080000:3.0:1775766244.363151:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766244.363154:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766244.363159:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000800:00000100:1.0:1775766244.363172:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766244.363176:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747480 x1862022114465280/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766244.363190:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:1.0:1775766244.363191:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766244/real 1775766244]  req@ffff9f1485747b80 x1862022114464640/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766269 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766244.363197:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775766244.363203:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766244.363222:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3621
00000100:00080000:3.0:1775766244.363230:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766244.363234:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766244.363238:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775766244.363244:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747b80 x1862022114464640/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766269 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766244.363248:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745880 x1862022114465408/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766244.363325:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775766244.363332:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775766244.363339:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114464640:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775766244.363350:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114464768:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775766244.363376:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775766244.363387:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766244/real 1775766244]  req@ffff9f1485744e00 x1862022114464768/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766269 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775766244.363400:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744e00 x1862022114464768/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766269 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766244.363408:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775766244.363411:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775766244.363414:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114464768:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775766244.363430:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114464896:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775766244.363452:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775766244.363463:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766244/real 1775766244]  req@ffff9f1485744380 x1862022114464896/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766269 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775766244.363473:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744380 x1862022114464896/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766269 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766244.363482:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775766244.363485:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775766244.363488:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114464896:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775766244.363494:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114465024:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775766244.363513:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775766244.363523:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766244/real 1775766244]  req@ffff9f1485746a00 x1862022114465024/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766269 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775766244.363535:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746a00 x1862022114465024/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766269 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766244.363543:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775766244.363546:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775766244.363549:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114465024:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775766244.363582:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114465152:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775766244.363601:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775766244.363609:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766244/real 1775766244]  req@ffff9f1485746300 x1862022114465152/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766269 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775766244.363618:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746300 x1862022114465152/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766269 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766244.363623:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775766244.363625:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775766244.363627:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114465152:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775766244.363666:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114465280:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775766244.363688:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775766244.363700:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766244/real 1775766244]  req@ffff9f1485747480 x1862022114465280/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766269 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775766244.363708:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747480 x1862022114465280/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766269 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766244.363718:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775766244.363721:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775766244.363723:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114465280:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775766244.363727:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114465408:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775766244.363740:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775766244.363746:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766244/real 1775766244]  req@ffff9f1485745880 x1862022114465408/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766269 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775766244.363754:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745880 x1862022114465408/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766269 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766244.363759:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775766244.363761:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775766244.363763:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114465408:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775766244.618714:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775766245.386609:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775766245.642735:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775766246.410762:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766246.666706:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766247.434708:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766247.690766:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766248.458690:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766248.714719:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1775766249.482645:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:2.0:1775766249.482651:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775766249.482656:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775766249.482662:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766249.482672:0:8:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3626
00000100:00080000:3.0:1775766249.482687:0:8:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766249.482694:0:8:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775766249.482742:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744a80 x1862022114465536/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766249.482772:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766249.482775:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775766249.482777:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766249.482780:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3626
00000100:00080000:3.0:1775766249.482782:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766249.482785:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766249.482795:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766249.482808:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745880 x1862022114465664/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766249.482817:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:1.0:1775766249.482818:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114465536:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775766249.482821:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775766249.482824:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766249.482830:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3626
00000100:00080000:3.0:1775766249.482832:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766249.482834:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766249.482837:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766249.482846:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747480 x1862022114465792/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766249.482853:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766249.482856:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775766249.482858:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766249.482860:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3626
00000100:00080000:3.0:1775766249.482863:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766249.482866:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766249.482869:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766249.482876:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746300 x1862022114465920/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766249.482882:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766249.482884:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775766249.482885:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766249.482887:0:8:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3626
00000100:00080000:3.0:1775766249.482891:0:8:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766249.482893:0:8:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766249.482897:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:1.0:1775766249.482902:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766249.482908:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746a00 x1862022114466048/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766249.482915:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766249.482917:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775766249.482919:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766249.482922:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3626
00000100:00080000:3.0:1775766249.482924:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766249.482928:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766249.482931:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766249.482938:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744380 x1862022114466176/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766249.482949:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766249.482951:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775766249.482953:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766249.482956:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3626
00000100:00080000:3.0:1775766249.482959:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:1.0:1775766249.482963:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766249/real 1775766249]  req@ffff9f1485744a80 x1862022114465536/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766274 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766249.482965:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766249.482971:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766249.482983:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744e00 x1862022114466304/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:1.0:1775766249.482989:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744a80 x1862022114465536/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766274 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766249.482989:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766249.482995:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775766249.482998:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766249.483006:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3626
00000100:00080000:3.0:1775766249.483010:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766249.483011:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775766249.483016:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775766249.483021:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114465536:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775766249.483024:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766249.483036:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775766249.483050:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114465664:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766249.483052:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747b80 x1862022114466432/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:1.0:1775766249.483077:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775766249.483089:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766249/real 1775766249]  req@ffff9f1485745880 x1862022114465664/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766274 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775766249.483097:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745880 x1862022114465664/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766274 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766249.483102:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775766249.483105:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775766249.483107:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114465664:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775766249.483112:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114465792:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775766249.483123:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775766249.483129:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766249/real 1775766249]  req@ffff9f1485747480 x1862022114465792/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766274 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775766249.483137:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747480 x1862022114465792/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766274 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766249.483142:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775766249.483143:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775766249.483145:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114465792:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775766249.483149:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114465920:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775766249.483161:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775766249.483167:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766249/real 1775766249]  req@ffff9f1485746300 x1862022114465920/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766274 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775766249.483174:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746300 x1862022114465920/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766274 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766249.483180:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775766249.483181:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775766249.483183:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114465920:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775766249.483186:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114466048:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775766249.483196:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775766249.483201:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766249/real 1775766249]  req@ffff9f1485746a00 x1862022114466048/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766274 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775766249.483209:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746a00 x1862022114466048/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766274 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766249.483214:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775766249.483216:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775766249.483217:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114466048:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775766249.483221:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114466176:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775766249.483230:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775766249.483236:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766249/real 1775766249]  req@ffff9f1485744380 x1862022114466176/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766274 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775766249.483249:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744380 x1862022114466176/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766274 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766249.483254:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775766249.483256:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775766249.483257:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114466176:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775766249.483260:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114466304:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775766249.483270:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775766249.483275:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766249/real 1775766249]  req@ffff9f1485744e00 x1862022114466304/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766274 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775766249.483281:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744e00 x1862022114466304/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766274 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766249.483286:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775766249.483288:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775766249.483290:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114466304:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775766249.483304:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114466432:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775766249.483313:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775766249.483318:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766249/real 1775766249]  req@ffff9f1485747b80 x1862022114466432/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766274 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775766249.483325:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747b80 x1862022114466432/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766274 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766249.483335:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775766249.483338:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775766249.483340:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114466432:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775766249.738789:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766250.506787:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775766250.762788:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766251.530661:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766251.786751:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766252.554732:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775766252.810768:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766253.578753:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766253.834763:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766254.602702:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:1.0:1775766254.602705:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766254.602714:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1775766254.602718:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766254.602729:0:8:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3632
00000100:00080000:1.0:1775766254.602742:0:8:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766254.602750:0:8:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775766254.602797:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff100 x1862022114466560/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766254.602826:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766254.602840:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775766254.602858:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766254.602861:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3632
00000100:00080000:1.0:1775766254.602864:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766254.602868:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766254.602879:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775766254.602890:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fd880 x1862022114466688/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766254.602900:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766254.602905:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775766254.602906:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766254.602928:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3632
00000100:00080000:1.0:1775766254.602934:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766254.602949:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775766254.602952:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114466560:192.168.202.110@tcp:250:kworker.0
00000008:00100000:1.0:1775766254.602954:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775766254.602968:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fe300 x1862022114466816/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766254.602975:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766254.602978:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775766254.602980:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766254.602984:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3632
00000100:00080000:1.0:1775766254.602987:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766254.602989:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766254.602993:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775766254.603000:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fc380 x1862022114466944/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766254.603012:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766254.603015:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1775766254.603017:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766254.603021:0:8:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3632
00000100:00080000:1.0:1775766254.603027:0:8:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766254.603032:0:8:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766254.603039:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:3.0:1775766254.603053:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775766254.603059:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fea00 x1862022114467072/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766254.603070:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766254.603075:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775766254.603077:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766254.603083:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3632
00000100:00080000:1.0:1775766254.603087:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766254.603092:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766254.603099:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766254.603109:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766254/real 1775766254]  req@ffff9f14ab4ff100 x1862022114466560/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766279 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775766254.603113:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fce00 x1862022114467200/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766254.603177:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766254.603183:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775766254.603185:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766254.603189:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3632
00000100:00080000:1.0:1775766254.603193:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766254.603197:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775766254.603203:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff100 x1862022114466560/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766279 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:1.0:1775766254.603209:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775766254.603223:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fca80 x1862022114467328/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766254.603224:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766254.603239:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:1.0:1775766254.603247:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:3.0:1775766254.603252:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114466560:192.168.202.110@tcp:250:kworker.0
00000100:00080000:1.0:1775766254.603256:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775766254.603265:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766254.603271:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3632
00000100:00100000:3.0:1775766254.603272:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114466688:192.168.202.110@tcp:38:kworker.0
00000100:00080000:1.0:1775766254.603297:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766254.603302:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766254.603307:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775766254.603318:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fed80 x1862022114467456/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:3.0:1775766254.603322:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766254.603334:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766254/real 1775766254]  req@ffff9f14ab4fd880 x1862022114466688/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766279 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766254.603346:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fd880 x1862022114466688/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766279 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766254.603355:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766254.603357:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766254.603360:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114466688:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766254.603380:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114466816:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775766254.603393:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766254.603399:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766254/real 1775766254]  req@ffff9f14ab4fe300 x1862022114466816/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766279 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766254.603415:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fe300 x1862022114466816/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766279 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766254.603426:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766254.603428:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766254.603430:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114466816:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766254.603435:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114466944:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775766254.603447:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766254.603453:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766254/real 1775766254]  req@ffff9f14ab4fc380 x1862022114466944/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766279 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766254.603460:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fc380 x1862022114466944/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766279 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766254.603466:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766254.603468:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766254.603470:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114466944:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775766254.603473:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114467072:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775766254.603482:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766254.603488:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766254/real 1775766254]  req@ffff9f14ab4fea00 x1862022114467072/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766279 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766254.603495:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fea00 x1862022114467072/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766279 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766254.603502:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766254.603506:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766254.603509:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114467072:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775766254.603515:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114467200:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775766254.603575:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766254.603590:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766254/real 1775766254]  req@ffff9f14ab4fce00 x1862022114467200/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766279 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766254.603603:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fce00 x1862022114467200/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766279 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766254.603612:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766254.603615:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766254.603619:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114467200:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766254.603625:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114467328:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775766254.603646:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766254.603657:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766254/real 1775766254]  req@ffff9f14ab4fca80 x1862022114467328/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766279 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766254.603670:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fca80 x1862022114467328/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766279 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766254.603678:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766254.603682:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766254.603685:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114467328:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766254.603691:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114467456:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775766254.603714:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766254.603726:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766254/real 1775766254]  req@ffff9f14ab4fed80 x1862022114467456/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766279 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766254.603738:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fed80 x1862022114467456/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766279 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766254.603747:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766254.603750:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766254.603754:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114467456:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775766254.858836:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766255.626687:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766255.882729:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766256.650660:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766256.906763:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775766257.674717:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775766257.930655:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775766258.698632:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766258.954719:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775766259.722674:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:2.0:1775766259.722687:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766259.722694:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1775766259.722699:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766259.722709:0:8:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3637
00000100:00080000:2.0:1775766259.722721:0:8:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766259.722729:0:8:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775766259.722768:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2a00 x1862022114467584/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766259.722796:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766259.722800:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775766259.722802:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766259.722806:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3637
00000100:00080000:2.0:1775766259.722809:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766259.722812:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766259.722822:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766259.722851:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1f80 x1862022114467712/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766259.722862:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766259.722865:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775766259.722867:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766259.722871:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3637
00000100:00080000:2.0:1775766259.722874:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766259.722880:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775766259.722881:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114467584:192.168.202.110@tcp:250:kworker.0
00000008:00100000:2.0:1775766259.722886:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766259.722897:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0000 x1862022114467840/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766259.722915:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766259.722918:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775766259.722920:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766259.722924:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3637
00000100:00080000:2.0:1775766259.722938:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766259.722942:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766259.722947:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:3.0:1775766259.722955:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766259.722960:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3b80 x1862022114467968/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766259.722971:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766259.722975:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1775766259.722977:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766259.722981:0:8:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3637
00000100:00080000:2.0:1775766259.722984:0:8:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766259.723004:0:8:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766259.723009:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766259.723022:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766259/real 1775766259]  req@ffff9f14852d2a00 x1862022114467584/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766284 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766259.723037:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0e00 x1862022114468096/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775766259.723047:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2a00 x1862022114467584/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766284 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766259.723048:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766259.723057:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775766259.723061:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766259.723071:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3637
00000100:00080000:3.0:1775766259.723073:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766259.723076:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766259.723081:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:2.0:1775766259.723086:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775766259.723090:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114467584:192.168.202.110@tcp:250:kworker.0
00000008:00100000:2.0:1775766259.723094:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766259.723101:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114467712:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766259.723108:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1880 x1862022114468224/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766259.723119:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766259.723122:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775766259.723125:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766259.723129:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3637
00000100:00080000:2.0:1775766259.723134:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766259.723137:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766259.723142:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000800:00000100:3.0:1775766259.723146:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766259.723156:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3100 x1862022114468352/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766259.723166:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766259.723172:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00100000:3.0:1775766259.723172:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766259/real 1775766259]  req@ffff9f14852d1f80 x1862022114467712/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766284 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766259.723178:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766259.723191:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3637
00000100:00080000:2.0:1775766259.723202:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:3.0:1775766259.723210:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1f80 x1862022114467712/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766284 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766259.723211:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766259.723224:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00080000:3.0:1775766259.723240:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766259.723246:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766259.723250:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2300 x1862022114468480/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775766259.723254:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114467712:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766259.723292:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114467840:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775766259.723316:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766259.723327:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766259/real 1775766259]  req@ffff9f14852d0000 x1862022114467840/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766284 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766259.723338:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0000 x1862022114467840/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766284 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766259.723346:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766259.723348:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766259.723351:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114467840:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766259.723356:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114467968:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775766259.723370:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766259.723378:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766259/real 1775766259]  req@ffff9f14852d3b80 x1862022114467968/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766284 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766259.723388:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3b80 x1862022114467968/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766284 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766259.723395:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766259.723397:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766259.723401:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114467968:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775766259.723406:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114468096:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775766259.723420:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766259.723427:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766259/real 1775766259]  req@ffff9f14852d0e00 x1862022114468096/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766284 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766259.723437:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0e00 x1862022114468096/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766284 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766259.723444:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766259.723447:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766259.723449:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114468096:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775766259.723454:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114468224:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775766259.723466:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766259.723473:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766259/real 1775766259]  req@ffff9f14852d1880 x1862022114468224/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766284 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766259.723503:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1880 x1862022114468224/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766284 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766259.723516:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766259.723521:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766259.723526:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114468224:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766259.723535:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114468352:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775766259.723601:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766259.723625:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766259/real 1775766259]  req@ffff9f14852d3100 x1862022114468352/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766284 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766259.723641:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3100 x1862022114468352/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766284 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766259.723649:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766259.723652:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766259.723654:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114468352:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766259.723660:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114468480:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775766259.723677:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766259.723686:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766259/real 1775766259]  req@ffff9f14852d2300 x1862022114468480/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766284 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766259.723696:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2300 x1862022114468480/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766284 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766259.723703:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766259.723705:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766259.723707:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114468480:192.168.202.110@tcp:8:kworker.0
00010000:00010000:3.0:1775766259.978801:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775766260.746636:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766261.002743:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775766261.770649:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766262.026799:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775766262.794618:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766263.050775:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775766263.818623:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766264.074704:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775766264.842654:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:1.0:1775766264.842724:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766264.842731:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1775766264.842734:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766264.842743:0:8:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3642
00000100:00080000:1.0:1775766264.842755:0:8:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766264.842762:0:8:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775766264.842795:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff800 x1862022114468608/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766264.842820:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766264.842823:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775766264.842825:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766264.842830:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3642
00000100:00080000:1.0:1775766264.842835:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766264.842839:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766264.842848:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766264.842876:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114468608:192.168.202.110@tcp:250:kworker.0
00000100:00100000:1.0:1775766264.842886:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ffb80 x1862022114468736/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766264.842900:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766264.842904:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775766264.842907:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766264.842913:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3642
00000100:00080000:1.0:1775766264.842934:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766264.842940:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766264.842947:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775766264.842964:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdf80 x1862022114468864/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766264.842978:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766264.842982:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775766264.842984:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766264.842989:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3642
00000100:00100000:2.0:1775766264.842994:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ffb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114468736:192.168.202.110@tcp:38:kworker.0
00000100:00080000:1.0:1775766264.842996:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766264.843004:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766264.843011:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775766264.843023:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114468864:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775766264.843030:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdc00 x1862022114468992/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766264.843046:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:2.0:1775766264.843048:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114468992:192.168.202.110@tcp:8:kworker.0
00000100:00080000:1.0:1775766264.843053:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1775766264.843056:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766264.843063:0:8:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3642
00000100:00080000:1.0:1775766264.843070:0:8:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766264.843076:0:8:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766264.843084:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775766264.843101:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fed80 x1862022114469120/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766264.843118:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766264.843123:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775766264.843125:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766264.843131:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3642
00000100:00100000:2.0:1775766264.843135:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114469120:192.168.202.110@tcp:8:kworker.0
00000100:00080000:1.0:1775766264.843139:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766264.843146:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766264.843154:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775766264.843171:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fca80 x1862022114469248/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766264.843191:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766264.843211:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775766264.843214:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766264.843221:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3642
00000100:00100000:2.0:1775766264.843227:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114469248:192.168.202.110@tcp:38:kworker.0
00000100:00080000:1.0:1775766264.843228:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766264.843239:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766264.843246:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775766264.843263:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fce00 x1862022114469376/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766264.843293:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766264.843297:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775766264.843300:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766264.843305:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3642
00000100:00080000:1.0:1775766264.843310:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:2.0:1775766264.843315:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114469376:192.168.202.110@tcp:38:kworker.0
00000100:00080000:1.0:1775766264.843318:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766264.843328:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775766264.843359:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fea00 x1862022114469504/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775766264.843386:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114469504:192.168.202.110@tcp:8:kworker.0
00010000:00010000:1.0:1775766265.098690:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775766265.866624:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766266.122714:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775766266.890652:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775766267.146781:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766267.914724:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000400:00000100:0.0:1775766267.914899:0:2392:0:(lib-socket.c:454:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.202.110:988
00000400:00000100:0.0:1775766267.914954:0:2392:0:(acceptor.c:111:lnet_connect_console_error()) Connection to 192.168.202.110@tcp at host 192.168.202.110 was unreachable: the network or that node may be down, or Lustre may be misconfigured.
00000800:00000100:0.0:1775766267.914979:0:2392:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:0.0:1775766267.915032:0:2392:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:0.0:1775766267.915047:0:2392:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:0.0:1775766267.915088:0:2392:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000100:00100000:2.0:1775766267.915107:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766264/real 1775766267]  req@ffff9f14ab4ffb80 x1862022114468736/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766289 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000800:00000100:0.0:1775766267.915108:0:2392:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:0.0:1775766267.915123:0:2392:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:0.0:1775766267.915145:0:2392:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:0.0:1775766267.915166:0:2392:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000100:00100000:2.0:1775766267.915184:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ffb80 x1862022114468736/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766289 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766267.915206:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766267.915215:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766267.915224:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ffb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114468736:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766267.915237:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766264/real 1775766267]  req@ffff9f14ab4fdf80 x1862022114468864/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766289 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766267.915261:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdf80 x1862022114468864/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766289 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766267.915284:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766267.915290:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766267.915297:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114468864:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766267.915308:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766264/real 1775766267]  req@ffff9f14ab4ff800 x1862022114468608/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766289 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766267.915332:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff800 x1862022114468608/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766289 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766267.915344:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766267.915351:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766267.915357:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114468608:192.168.202.110@tcp:250:kworker.0
00000100:00100000:2.0:1775766267.915396:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766264/real 1775766267]  req@ffff9f14ab4fea00 x1862022114469504/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766289 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766267.915450:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fea00 x1862022114469504/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766289 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766267.915465:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766267.915472:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766267.915480:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114469504:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775766267.915491:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766264/real 1775766267]  req@ffff9f14ab4fce00 x1862022114469376/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766289 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766267.915513:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fce00 x1862022114469376/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766289 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766267.915526:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766267.915532:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766267.915547:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114469376:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766267.915595:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766264/real 1775766267]  req@ffff9f14ab4fca80 x1862022114469248/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766289 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766267.915628:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fca80 x1862022114469248/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766289 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766267.915642:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766267.915648:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766267.915654:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114469248:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766267.915665:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766264/real 1775766267]  req@ffff9f14ab4fed80 x1862022114469120/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766289 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766267.915685:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fed80 x1862022114469120/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766289 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766267.915698:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766267.915704:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766267.915710:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114469120:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775766267.915721:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766264/real 1775766267]  req@ffff9f14ab4fdc00 x1862022114468992/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766289 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766267.915753:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdc00 x1862022114468992/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766289 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766267.915767:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766267.915773:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766267.915779:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114468992:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775766268.170853:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766268.938619:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766269.194732:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1775766269.962637:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:2.0:1775766269.962639:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775766269.962646:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775766269.962650:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766269.962659:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3647
00000100:00080000:3.0:1775766269.962669:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766269.962677:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775766269.962710:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746680 x1862022114469632/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766269.962731:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766269.962735:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775766269.962737:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766269.962742:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3647
00000100:00080000:3.0:1775766269.962746:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766269.962750:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766269.962760:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766269.962763:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114469632:192.168.202.110@tcp:250:kworker.0
00000100:00100000:3.0:1775766269.962771:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744700 x1862022114469760/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766269.962781:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766269.962784:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775766269.962785:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766269.962789:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3647
00000100:00080000:3.0:1775766269.962791:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766269.962794:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766269.962798:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766269.962807:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745500 x1862022114469888/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766269.962814:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766269.962817:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775766269.962819:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000800:00000100:2.0:1775766269.962820:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775766269.962822:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3647
00000100:00080000:3.0:1775766269.962825:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766269.962829:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766269.962833:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766269.962841:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744000 x1862022114470016/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766269.962849:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766269.962852:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775766269.962854:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:2.0:1775766269.962858:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766269/real 1775766269]  req@ffff9f1485746680 x1862022114469632/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766294 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766269.962858:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3647
00000100:00080000:3.0:1775766269.962861:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766269.962865:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766269.962869:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775766269.962876:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746680 x1862022114469632/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766294 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766269.962877:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747800 x1862022114470144/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766269.962886:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766269.962887:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766269.962890:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:3.0:1775766269.962891:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775766269.962892:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:2.0:1775766269.962895:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114469632:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775766269.962896:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3647
00000100:00080000:3.0:1775766269.962899:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766269.962902:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766269.962906:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766269.962914:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745c00 x1862022114470272/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775766269.962915:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114469760:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775766269.962923:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766269.962926:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775766269.962928:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766269.962931:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3647
00000100:00080000:3.0:1775766269.962935:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000800:00000100:2.0:1775766269.962939:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775766269.962939:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766269.962944:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766269.962950:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766269/real 1775766269]  req@ffff9f1485744700 x1862022114469760/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766294 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766269.962953:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745180 x1862022114470400/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766269.962962:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:2.0:1775766269.962964:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744700 x1862022114469760/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766294 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766269.962966:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775766269.962968:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766269.962972:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766269.962972:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3647
00000100:00080000:3.0:1775766269.962975:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766269.962976:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766269.962979:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114469760:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775766269.962979:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766269.962983:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775766269.962985:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114469888:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766269.962992:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745f80 x1862022114470528/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:2.0:1775766269.963002:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766269.963011:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766269/real 1775766269]  req@ffff9f1485745500 x1862022114469888/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766294 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766269.963023:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745500 x1862022114469888/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766294 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766269.963031:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766269.963035:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766269.963038:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114469888:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766269.963044:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114470016:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775766269.963066:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766269.963076:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766269/real 1775766269]  req@ffff9f1485744000 x1862022114470016/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766294 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766269.963095:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744000 x1862022114470016/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766294 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766269.963102:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766269.963112:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766269.963115:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114470016:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775766269.963121:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114470144:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775766269.963140:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766269.963150:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766269/real 1775766269]  req@ffff9f1485747800 x1862022114470144/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766294 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766269.963161:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747800 x1862022114470144/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766294 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766269.963169:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766269.963172:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766269.963182:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114470144:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775766269.963200:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114470272:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775766269.963221:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766269.963232:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766269/real 1775766269]  req@ffff9f1485745c00 x1862022114470272/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766294 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766269.963244:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745c00 x1862022114470272/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766294 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766269.963258:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766269.963262:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766269.963265:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114470272:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766269.963270:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114470400:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775766269.963290:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766269.963301:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766269/real 1775766269]  req@ffff9f1485745180 x1862022114470400/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766294 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766269.963313:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745180 x1862022114470400/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766294 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766269.963320:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766269.963323:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766269.963326:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114470400:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766269.963333:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114470528:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775766269.963352:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766269.963363:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766269/real 1775766269]  req@ffff9f1485745f80 x1862022114470528/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766294 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766269.963375:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745f80 x1862022114470528/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766294 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766269.963383:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766269.963400:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766269.963405:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114470528:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775766270.218808:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766270.986678:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766271.242714:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766272.010670:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775766272.266757:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766273.034662:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766273.290695:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766274.058641:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775766274.314723:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766275.082631:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775766275.082631:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766275.082640:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775766275.082645:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766275.082653:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3652
00000100:00080000:3.0:1775766275.082662:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766275.082668:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775766275.082701:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747100 x1862022114470656/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766275.082724:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766275.082727:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775766275.082728:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766275.082731:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3652
00000100:00080000:3.0:1775766275.082733:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766275.082736:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766275.082744:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766275.082756:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114470656:192.168.202.110@tcp:250:kworker.0
00000100:00100000:3.0:1775766275.082756:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745f80 x1862022114470784/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766275.082770:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766275.082772:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775766275.082773:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766275.082776:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3652
00000100:00080000:3.0:1775766275.082778:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766275.082780:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766275.082783:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766275.082793:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745180 x1862022114470912/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766275.082803:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766275.082806:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775766275.082808:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766275.082810:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3652
00000100:00080000:3.0:1775766275.082812:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766275.082814:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766275.082818:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:2.0:1775766275.082821:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766275.082830:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745c00 x1862022114471040/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766275.082837:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766275.082840:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775766275.082841:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766275.082843:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3652
00000100:00080000:3.0:1775766275.082845:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766275.082848:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766275.082851:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775766275.082860:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766275/real 1775766275]  req@ffff9f1485747100 x1862022114470656/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766300 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766275.082862:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747800 x1862022114471168/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766275.082887:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766275.082889:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775766275.082890:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:2.0:1775766275.082893:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747100 x1862022114470656/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766300 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766275.082897:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3652
00000100:00080000:3.0:1775766275.082903:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766275.082909:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00080000:2.0:1775766275.082910:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000008:00100000:3.0:1775766275.082912:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00080000:2.0:1775766275.082916:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766275.082919:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114470656:192.168.202.110@tcp:250:kworker.0
00000100:00100000:3.0:1775766275.082924:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744000 x1862022114471296/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766275.082931:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766275.082934:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775766275.082935:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766275.082937:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3652
00000100:00080000:3.0:1775766275.082940:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:2.0:1775766275.082943:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114470784:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775766275.082946:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766275.082951:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766275.082961:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745500 x1862022114471424/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:2.0:1775766275.082967:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775766275.082968:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766275.082971:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775766275.082973:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766275.082977:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3652
00000100:00100000:2.0:1775766275.082980:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766275/real 1775766275]  req@ffff9f1485745f80 x1862022114470784/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766300 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766275.082982:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766275.082989:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766275.082996:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775766275.083001:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745f80 x1862022114470784/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766300 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766275.083006:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744700 x1862022114471552/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766275.083022:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766275.083025:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766275.083028:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114470784:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766275.083039:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114470912:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775766275.083059:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766275.083067:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766275/real 1775766275]  req@ffff9f1485745180 x1862022114470912/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766300 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766275.083075:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745180 x1862022114470912/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766300 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766275.083080:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766275.083082:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766275.083084:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114470912:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766275.083088:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114471040:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775766275.083098:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766275.083103:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766275/real 1775766275]  req@ffff9f1485745c00 x1862022114471040/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766300 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766275.083110:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745c00 x1862022114471040/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766300 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766275.083115:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766275.083117:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766275.083119:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114471040:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775766275.083123:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114471168:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775766275.083132:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766275.083137:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766275/real 1775766275]  req@ffff9f1485747800 x1862022114471168/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766300 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766275.083144:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747800 x1862022114471168/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766300 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766275.083149:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766275.083151:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766275.083152:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114471168:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775766275.083156:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114471296:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775766275.083165:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766275.083170:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766275/real 1775766275]  req@ffff9f1485744000 x1862022114471296/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766300 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766275.083177:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744000 x1862022114471296/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766300 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766275.083182:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766275.083184:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766275.083185:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114471296:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766275.083196:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114471424:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775766275.083205:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766275.083211:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766275/real 1775766275]  req@ffff9f1485745500 x1862022114471424/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766300 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766275.083229:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745500 x1862022114471424/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766300 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766275.083235:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766275.083236:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766275.083238:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114471424:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766275.083242:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114471552:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775766275.083251:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766275.083257:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766275/real 1775766275]  req@ffff9f1485744700 x1862022114471552/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766300 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766275.083269:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744700 x1862022114471552/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766300 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766275.083277:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766275.083280:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766275.083295:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114471552:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775766275.338757:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766276.106629:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766276.362769:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766277.130665:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766277.386740:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766278.154658:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766278.410754:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766279.178657:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775766279.434718:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766280.202725:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:1.0:1775766280.202770:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766280.202783:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1775766280.202788:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766280.202800:0:8:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3657
00000100:00080000:1.0:1775766280.202831:0:8:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766280.202840:0:8:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775766280.202883:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fc380 x1862022114471680/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766280.202913:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766280.202916:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775766280.202918:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766280.202922:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3657
00000100:00080000:1.0:1775766280.202927:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766280.202932:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766280.202942:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766280.202954:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114471680:192.168.202.110@tcp:250:kworker.0
00000100:00100000:1.0:1775766280.202986:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fe300 x1862022114471808/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766280.203000:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766280.203003:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775766280.203005:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766280.203009:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3657
00000100:00080000:1.0:1775766280.203013:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766280.203017:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766280.203023:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775766280.203033:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fd880 x1862022114471936/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766280.203043:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766280.203046:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775766280.203048:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766280.203052:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3657
00000100:00080000:1.0:1775766280.203056:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000800:00000100:2.0:1775766280.203061:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:1.0:1775766280.203065:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766280.203069:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775766280.203079:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff100 x1862022114472064/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766280.203087:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766280.203091:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1775766280.203092:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766280.203098:0:8:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3657
00000100:00080000:1.0:1775766280.203101:0:8:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766280.203105:0:8:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766280.203109:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775766280.203121:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdc00 x1862022114472192/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766280.203130:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:2.0:1775766280.203132:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766280/real 1775766280]  req@ffff9f14ab4fc380 x1862022114471680/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766305 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766280.203134:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775766280.203137:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766280.203154:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3657
00000100:00080000:1.0:1775766280.203169:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766280.203174:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766280.203179:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766280.203181:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fc380 x1862022114471680/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766305 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775766280.203193:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fed80 x1862022114472320/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766280.203195:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766280.203210:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766280.203225:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114471680:192.168.202.110@tcp:250:kworker.0
00000100:00080000:1.0:1775766280.203229:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766280.203233:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775766280.203236:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766280.203242:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3657
00000100:00080000:1.0:1775766280.203245:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766280.203247:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766280.203256:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766280.203264:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114471808:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775766280.203268:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fca80 x1862022114472448/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766280.203316:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000800:00000100:2.0:1775766280.203319:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:1.0:1775766280.203330:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775766280.203334:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:2.0:1775766280.203356:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766280/real 1775766280]  req@ffff9f14ab4fe300 x1862022114471808/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766305 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766280.203361:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3657
00000100:00080000:1.0:1775766280.203374:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766280.203382:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775766280.203389:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fe300 x1862022114471808/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766305 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:1.0:1775766280.203393:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00080000:2.0:1775766280.203407:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00100000:1.0:1775766280.203412:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fce00 x1862022114472576/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766280.203420:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766280.203437:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114471808:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766280.203448:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114471936:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775766280.203474:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766280.203483:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766280/real 1775766280]  req@ffff9f14ab4fd880 x1862022114471936/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766305 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766280.203501:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fd880 x1862022114471936/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766305 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766280.203507:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766280.203509:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766280.203512:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114471936:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766280.203517:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114472064:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775766280.203531:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766280.203538:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766280/real 1775766280]  req@ffff9f14ab4ff100 x1862022114472064/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766305 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766280.203546:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff100 x1862022114472064/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766305 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766280.203552:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766280.203577:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766280.203580:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114472064:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775766280.203584:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114472192:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775766280.203597:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766280.203604:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766280/real 1775766280]  req@ffff9f14ab4fdc00 x1862022114472192/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766305 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766280.203623:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdc00 x1862022114472192/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766305 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766280.203633:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766280.203636:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766280.203638:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114472192:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775766280.203641:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114472320:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775766280.203652:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766280.203658:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766280/real 1775766280]  req@ffff9f14ab4fed80 x1862022114472320/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766305 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766280.203666:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fed80 x1862022114472320/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766305 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766280.203672:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766280.203674:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766280.203675:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114472320:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766280.203692:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114472448:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775766280.203703:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766280.203709:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766280/real 1775766280]  req@ffff9f14ab4fca80 x1862022114472448/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766305 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766280.203715:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fca80 x1862022114472448/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766305 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766280.203720:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766280.203722:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766280.203725:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114472448:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766280.203729:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114472576:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775766280.203740:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766280.203745:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766280/real 1775766280]  req@ffff9f14ab4fce00 x1862022114472576/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766305 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766280.203753:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fce00 x1862022114472576/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766305 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766280.203758:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766280.203760:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766280.203762:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114472576:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775766280.458876:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766281.226703:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775766281.482697:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766282.250724:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766282.506766:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766283.274662:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766283.530719:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766284.298647:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766284.554711:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1775766285.322645:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:2.0:1775766285.322651:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:1.0:1775766285.322654:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1775766285.322659:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766285.322670:0:8:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3662
00000100:00080000:1.0:1775766285.322681:0:8:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766285.322695:0:8:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775766285.322732:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fea00 x1862022114472704/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766285.322755:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766285.322758:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775766285.322760:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766285.322763:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3662
00000100:00080000:1.0:1775766285.322766:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766285.322769:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766285.322777:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775766285.322788:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff800 x1862022114472832/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766285.322798:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766285.322802:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775766285.322804:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:2.0:1775766285.322805:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114472704:192.168.202.110@tcp:250:kworker.0
00000100:00080000:1.0:1775766285.322810:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3662
00000100:00080000:1.0:1775766285.322816:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766285.322820:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766285.322823:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775766285.322832:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdf80 x1862022114472960/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766285.322845:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766285.322848:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775766285.322849:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766285.322852:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3662
00000100:00080000:1.0:1775766285.322854:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766285.322863:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766285.322867:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775766285.322886:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ffb80 x1862022114473088/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766285.322894:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766285.322897:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1775766285.322898:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766285.322900:0:8:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3662
00000100:00080000:1.0:1775766285.322903:0:8:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766285.322906:0:8:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766285.322910:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:2.0:1775766285.322912:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775766285.322922:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fce00 x1862022114473216/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766285.322930:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766285.322933:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775766285.322935:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766285.322949:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3662
00000100:00080000:1.0:1775766285.322952:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766285.322955:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766285.322958:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775766285.322967:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fca80 x1862022114473344/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766285.322975:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766285.322979:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775766285.322980:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:2.0:1775766285.322980:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766285/real 1775766285]  req@ffff9f14ab4fea00 x1862022114472704/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766310 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766285.322987:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3662
00000100:00080000:1.0:1775766285.322993:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766285.322999:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766285.323012:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775766285.323023:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fed80 x1862022114473472/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775766285.323030:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fea00 x1862022114472704/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766310 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766285.323032:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766285.323040:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775766285.323043:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766285.323052:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3662
00000100:00080000:1.0:1775766285.323059:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766285.323062:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00080000:2.0:1775766285.323063:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000008:00100000:1.0:1775766285.323065:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00080000:2.0:1775766285.323073:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775766285.323076:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdc00 x1862022114473600/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775766285.323083:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114472704:192.168.202.110@tcp:250:kworker.0
00000100:00100000:2.0:1775766285.323122:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114472832:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775766285.323154:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766285.323171:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766285/real 1775766285]  req@ffff9f14ab4ff800 x1862022114472832/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766310 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766285.323199:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff800 x1862022114472832/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766310 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766285.323219:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766285.323224:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766285.323228:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114472832:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766285.323236:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114472960:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775766285.323372:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766285.323387:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766285/real 1775766285]  req@ffff9f14ab4fdf80 x1862022114472960/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766310 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766285.323403:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdf80 x1862022114472960/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766310 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766285.323413:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766285.323417:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766285.323422:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114472960:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766285.323430:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ffb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114473088:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775766285.323469:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766285.323483:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766285/real 1775766285]  req@ffff9f14ab4ffb80 x1862022114473088/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766310 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766285.323509:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ffb80 x1862022114473088/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766310 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766285.323536:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766285.323541:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766285.323546:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ffb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114473088:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775766285.323555:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114473216:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775766285.323621:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766285.323638:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766285/real 1775766285]  req@ffff9f14ab4fce00 x1862022114473216/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766310 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766285.323657:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fce00 x1862022114473216/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766310 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766285.323669:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766285.323674:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766285.323679:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114473216:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775766285.323688:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114473344:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775766285.323719:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766285.323749:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766285/real 1775766285]  req@ffff9f14ab4fca80 x1862022114473344/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766310 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766285.323764:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fca80 x1862022114473344/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766310 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766285.323775:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766285.323779:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766285.323782:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114473344:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766285.323789:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114473472:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775766285.323831:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766285.323853:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766285/real 1775766285]  req@ffff9f14ab4fed80 x1862022114473472/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766310 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766285.323870:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fed80 x1862022114473472/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766310 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766285.323882:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766285.323902:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766285.323906:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114473472:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766285.323914:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114473600:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775766285.323947:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766285.323977:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766285/real 1775766285]  req@ffff9f14ab4fdc00 x1862022114473600/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766310 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766285.323996:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdc00 x1862022114473600/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766310 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766285.324018:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766285.324023:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766285.324028:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114473600:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775766285.578763:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775766286.346684:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766286.602736:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775766287.370681:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766287.626716:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766288.394680:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775766288.650845:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766289.418687:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766289.674712:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766290.442608:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:1.0:1775766290.442618:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766290.442625:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1775766290.442630:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766290.442639:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3667
00000100:00080000:1.0:1775766290.442651:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766290.442659:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775766290.442697:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff100 x1862022114473728/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766290.442720:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766290.442723:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775766290.442724:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766290.442727:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3667
00000100:00080000:1.0:1775766290.442730:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766290.442734:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766290.442742:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766290.442750:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114473728:192.168.202.110@tcp:250:kworker.0
00000100:00100000:1.0:1775766290.442755:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fd880 x1862022114473856/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766290.442768:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766290.442771:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775766290.442773:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766290.442778:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3667
00000100:00080000:1.0:1775766290.442783:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766290.442788:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766290.442793:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775766290.442807:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fe300 x1862022114473984/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:2.0:1775766290.442811:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:1.0:1775766290.442821:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766290.442823:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775766290.442825:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766290.442829:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3667
00000100:00080000:1.0:1775766290.442833:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766290.442845:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766290.442850:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775766290.442862:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fc380 x1862022114474112/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775766290.442864:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766290/real 1775766290]  req@ffff9f14ab4ff100 x1862022114473728/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766315 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766290.442877:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766290.442884:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1775766290.442887:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766290.442892:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3667
00000100:00080000:1.0:1775766290.442894:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766290.442897:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766290.442900:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775766290.442905:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff100 x1862022114473728/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766315 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775766290.442909:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdc00 x1862022114474240/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766290.442922:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766290.442930:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:1.0:1775766290.442933:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:2.0:1775766290.442936:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114473728:192.168.202.110@tcp:250:kworker.0
00000100:00080000:1.0:1775766290.442938:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775766290.442942:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766290.442945:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3667
00000100:00080000:1.0:1775766290.442948:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766290.442950:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766290.442953:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775766290.442960:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fed80 x1862022114474368/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775766290.442962:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114473856:192.168.202.110@tcp:38:kworker.0
00000100:00080000:1.0:1775766290.442972:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766290.442975:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775766290.442978:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766290.442982:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3667
00000100:00080000:1.0:1775766290.442985:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766290.442987:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766290.442990:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775766290.442998:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fca80 x1862022114474496/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:2.0:1775766290.443003:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:1.0:1775766290.443012:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766290.443015:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775766290.443016:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:2.0:1775766290.443020:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766290/real 1775766290]  req@ffff9f14ab4fd880 x1862022114473856/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766315 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766290.443022:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3667
00000100:00080000:1.0:1775766290.443028:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766290.443034:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766290.443041:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775766290.443052:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fd880 x1862022114473856/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766315 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775766290.443054:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fce00 x1862022114474624/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766290.443088:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766290.443095:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766290.443101:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114473856:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766290.443109:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114473984:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775766290.443129:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766290.443138:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766290/real 1775766290]  req@ffff9f14ab4fe300 x1862022114473984/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766315 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766290.443146:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fe300 x1862022114473984/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766315 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766290.443152:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766290.443154:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766290.443156:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114473984:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766290.443160:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114474112:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775766290.443171:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766290.443176:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766290/real 1775766290]  req@ffff9f14ab4fc380 x1862022114474112/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766315 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766290.443184:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fc380 x1862022114474112/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766315 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766290.443212:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766290.443215:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766290.443217:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114474112:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775766290.443221:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114474240:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775766290.443235:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766290.443242:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766290/real 1775766290]  req@ffff9f14ab4fdc00 x1862022114474240/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766315 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766290.443249:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdc00 x1862022114474240/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766315 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766290.443254:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766290.443257:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766290.443258:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114474240:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775766290.443270:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114474368:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775766290.443280:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766290.443286:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766290/real 1775766290]  req@ffff9f14ab4fed80 x1862022114474368/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766315 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766290.443293:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fed80 x1862022114474368/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766315 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766290.443298:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766290.443300:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766290.443302:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114474368:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766290.443306:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114474496:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775766290.443315:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766290.443320:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766290/real 1775766290]  req@ffff9f14ab4fca80 x1862022114474496/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766315 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766290.443327:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fca80 x1862022114474496/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766315 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766290.443332:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766290.443334:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766290.443335:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114474496:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766290.443338:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114474624:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775766290.443347:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766290.443352:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766290/real 1775766290]  req@ffff9f14ab4fce00 x1862022114474624/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766315 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766290.443358:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fce00 x1862022114474624/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766315 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766290.443363:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766290.443365:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766290.443367:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114474624:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775766290.698699:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766291.466623:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766291.722720:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766292.490650:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766292.746704:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775766293.514678:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766293.770677:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775766294.538676:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766294.794701:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775766295.562679:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:0.0:1775766295.562712:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766295.562722:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1775766295.562728:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766295.562743:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3673
00000100:00080000:0.0:1775766295.562757:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766295.562766:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775766295.562828:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d2a00 x1862022114474752/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775766295.562864:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766295.562869:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775766295.562872:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766295.562878:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3673
00000100:00080000:0.0:1775766295.562885:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766295.562892:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775766295.562902:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114474752:192.168.202.110@tcp:250:kworker.0
00000008:00100000:0.0:1775766295.562905:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775766295.562928:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0380 x1862022114474880/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775766295.562945:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766295.562950:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:0.0:1775766295.562953:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766295.562959:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3673
00000100:00080000:0.0:1775766295.562964:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766295.562970:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775766295.562977:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000800:00000100:2.0:1775766295.562984:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766295.562999:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d1f80 x1862022114475008/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775766295.563023:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766295.563029:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00100000:2.0:1775766295.563031:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766295/real 1775766295]  req@ffff9f14a04d2a00 x1862022114474752/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766320 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766295.563033:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766295.563045:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3673
00000100:00080000:0.0:1775766295.563056:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766295.563062:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775766295.563086:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775766295.563101:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d2a00 x1862022114474752/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766320 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766295.563108:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0a80 x1862022114475136/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766295.563134:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766295.563163:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766295.563185:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1775766295.563188:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766295.563193:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:0.0:1775766295.563197:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3673
00000100:00100000:2.0:1775766295.563205:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114474752:192.168.202.110@tcp:250:kworker.0
00000100:00080000:0.0:1775766295.563207:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766295.563219:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775766295.563227:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775766295.563243:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114474880:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766295.563248:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3b80 x1862022114475264/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775766295.563269:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766295.563274:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775766295.563277:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000800:00000100:2.0:1775766295.563282:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:0.0:1775766295.563290:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3673
00000100:00080000:0.0:1775766295.563298:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:2.0:1775766295.563301:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766295/real 1775766295]  req@ffff9f14a04d0380 x1862022114474880/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766320 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766295.563310:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775766295.563322:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766295.563323:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0380 x1862022114474880/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766320 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766295.563355:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766295.563360:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766295.563366:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114474880:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766295.563368:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d2a00 x1862022114475392/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775766295.563384:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114475008:192.168.202.110@tcp:38:kworker.0
00000100:00080000:0.0:1775766295.563402:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766295.563407:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:0.0:1775766295.563411:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000800:00000100:2.0:1775766295.563412:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:0.0:1775766295.563424:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3673
00000100:00100000:2.0:1775766295.563429:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766295/real 1775766295]  req@ffff9f14a04d1f80 x1862022114475008/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766320 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766295.563434:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766295.563445:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775766295.563447:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1f80 x1862022114475008/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766320 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:0.0:1775766295.563459:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00080000:2.0:1775766295.563461:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766295.563471:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766295.563476:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114475008:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766295.563485:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3100 x1862022114475520/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775766295.563505:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114475136:192.168.202.110@tcp:8:kworker.0
00000100:00080000:0.0:1775766295.563527:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766295.563532:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775766295.563536:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766295.563542:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3673
00000800:00000100:2.0:1775766295.563548:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:0.0:1775766295.563553:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766295.563561:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775766295.563568:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766295/real 1775766295]  req@ffff9f14a04d0a80 x1862022114475136/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766320 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:0.0:1775766295.563570:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775766295.563595:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0e00 x1862022114475648/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775766295.563621:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0a80 x1862022114475136/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766320 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766295.563630:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766295.563634:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766295.563643:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114475136:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775766295.563681:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114475264:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775766295.563708:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766295.563721:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766295/real 1775766295]  req@ffff9f14a04d3b80 x1862022114475264/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766320 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766295.563733:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3b80 x1862022114475264/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766320 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766295.563739:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766295.563742:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766295.563744:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114475264:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775766295.563749:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114475392:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775766295.563762:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766295.563768:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766295/real 1775766295]  req@ffff9f14a04d2a00 x1862022114475392/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766320 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766295.563778:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d2a00 x1862022114475392/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766320 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766295.563784:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766295.563787:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766295.563789:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114475392:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766295.563793:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114475520:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775766295.563805:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766295.563812:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766295/real 1775766295]  req@ffff9f14a04d3100 x1862022114475520/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766320 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766295.563838:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3100 x1862022114475520/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766320 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766295.563849:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766295.563852:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766295.563854:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114475520:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766295.563865:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114475648:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775766295.563878:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766295.563885:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766295/real 1775766295]  req@ffff9f14a04d0e00 x1862022114475648/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766320 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766295.563893:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0e00 x1862022114475648/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766320 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766295.563900:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766295.563902:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766295.563904:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114475648:192.168.202.110@tcp:8:kworker.0
00010000:00010000:3.0:1775766295.818744:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775766296.586694:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766296.842766:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775766297.610676:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766297.866790:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766298.634706:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766298.890787:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766299.658614:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766299.914700:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766300.682697:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:0.0:1775766300.682720:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766300.682733:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1775766300.682740:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766300.682756:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3678
00000100:00080000:0.0:1775766300.682771:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766300.682781:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775766300.682840:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d2300 x1862022114475776/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775766300.682905:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766300.682912:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775766300.682916:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766300.682926:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3678
00000100:00080000:0.0:1775766300.682934:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766300.682943:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775766300.682955:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114475776:192.168.202.110@tcp:250:kworker.0
00000008:00100000:0.0:1775766300.682960:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775766300.682990:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3480 x1862022114475904/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775766300.683029:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766300.683036:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:0.0:1775766300.683039:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766300.683049:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3678
00000100:00080000:0.0:1775766300.683058:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000800:00000100:2.0:1775766300.683060:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:0.0:1775766300.683072:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775766300.683085:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775766300.683156:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0000 x1862022114476032/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775766300.683168:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766300/real 1775766300]  req@ffff9f14a04d2300 x1862022114475776/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766325 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766300.683183:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766300.683193:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775766300.683197:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766300.683211:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3678
00000100:00080000:0.0:1775766300.683216:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766300.683228:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775766300.683238:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d2300 x1862022114475776/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766325 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:0.0:1775766300.683239:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00080000:2.0:1775766300.683259:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766300.683269:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766300.683270:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d1500 x1862022114476160/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775766300.683278:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114475776:192.168.202.110@tcp:250:kworker.0
00000100:00080000:0.0:1775766300.683304:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766300.683309:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1775766300.683312:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:2.0:1775766300.683318:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114475904:192.168.202.110@tcp:38:kworker.0
00000100:00080000:0.0:1775766300.683321:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3678
00000100:00080000:0.0:1775766300.683332:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766300.683339:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775766300.683348:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:2.0:1775766300.683349:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766300.683365:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766300/real 1775766300]  req@ffff9f14a04d3480 x1862022114475904/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766325 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766300.683375:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3800 x1862022114476288/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775766300.683412:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3480 x1862022114475904/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766325 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766300.683413:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766300.683427:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775766300.683433:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766300.683446:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3678
00000100:00080000:2.0:1775766300.683447:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766300.683456:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766300.683460:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00080000:0.0:1775766300.683470:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775766300.683471:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114475904:192.168.202.110@tcp:38:kworker.0
00000008:00100000:0.0:1775766300.683488:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766300.683488:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114476032:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766300.683523:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d1180 x1862022114476416/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:2.0:1775766300.683532:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:0.0:1775766300.683567:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:2.0:1775766300.683580:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766300/real 1775766300]  req@ffff9f14a04d0000 x1862022114476032/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766325 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766300.683592:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0000 x1862022114476032/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766325 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766300.683614:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766300.683616:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766300.683618:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114476032:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766300.683624:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114476160:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775766300.683648:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:0.0:1775766300.683657:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:0.0:1775766300.683662:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:2.0:1775766300.683664:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766300/real 1775766300]  req@ffff9f14a04d1500 x1862022114476160/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766325 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766300.683694:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3678
00000100:00080000:0.0:1775766300.683701:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:2.0:1775766300.683703:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1500 x1862022114476160/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766325 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766300.683715:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775766300.683732:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00080000:2.0:1775766300.683734:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766300.683741:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766300.683745:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114476160:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775766300.683759:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d2300 x1862022114476544/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775766300.683769:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114476288:192.168.202.110@tcp:8:kworker.0
00000100:00080000:0.0:1775766300.683771:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766300.683779:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775766300.683781:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766300.683786:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3678
00000100:00080000:0.0:1775766300.683792:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766300.683798:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000800:00000100:2.0:1775766300.683805:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000008:00100000:0.0:1775766300.683813:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775766300.683822:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766300/real 1775766300]  req@ffff9f14a04d3800 x1862022114476288/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766325 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766300.683831:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0e00 x1862022114476672/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775766300.683851:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3800 x1862022114476288/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766325 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766300.683864:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766300.683867:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766300.683870:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114476288:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775766300.683878:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114476416:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775766300.683902:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766300.683912:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766300/real 1775766300]  req@ffff9f14a04d1180 x1862022114476416/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766325 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766300.683920:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1180 x1862022114476416/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766325 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766300.683926:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766300.683929:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766300.683932:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114476416:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766300.683943:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114476544:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775766300.683983:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766300.683991:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766300/real 1775766300]  req@ffff9f14a04d2300 x1862022114476544/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766325 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766300.684000:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d2300 x1862022114476544/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766325 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766300.684006:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766300.684008:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766300.684010:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114476544:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766300.684014:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114476672:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775766300.684029:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766300.684037:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766300/real 1775766300]  req@ffff9f14a04d0e00 x1862022114476672/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766325 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766300.684046:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0e00 x1862022114476672/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766325 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766300.684052:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766300.684054:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766300.684057:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114476672:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775766300.938699:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766301.706655:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766301.962749:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766302.730662:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775766302.986710:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766303.754642:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766304.010703:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766304.778671:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766305.034690:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1775766305.802646:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:2.0:1775766305.802646:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:0.0:1775766305.802655:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1775766305.802659:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766305.802666:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3683
00000100:00080000:0.0:1775766305.802676:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766305.802681:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775766305.802715:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3100 x1862022114476800/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775766305.802738:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766305.802741:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775766305.802742:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766305.802745:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3683
00000100:00080000:0.0:1775766305.802747:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766305.802750:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775766305.802757:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775766305.802766:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d2a00 x1862022114476928/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775766305.802775:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114476800:192.168.202.110@tcp:250:kworker.0
00000100:00080000:0.0:1775766305.802776:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766305.802784:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:0.0:1775766305.802786:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766305.802790:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3683
00000100:00080000:0.0:1775766305.802792:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766305.802795:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775766305.802798:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775766305.802807:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3b80 x1862022114477056/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775766305.802813:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766305.802815:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775766305.802817:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766305.802819:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3683
00000100:00080000:0.0:1775766305.802821:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766305.802824:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775766305.802827:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775766305.802832:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0a80 x1862022114477184/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775766305.802838:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766305.802840:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1775766305.802841:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766305.802843:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3683
00000100:00080000:0.0:1775766305.802846:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000800:00000100:2.0:1775766305.802851:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:0.0:1775766305.802853:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775766305.802857:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775766305.802864:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d1f80 x1862022114477312/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775766305.802871:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766305.802874:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775766305.802875:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766305.802878:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3683
00000100:00080000:0.0:1775766305.802880:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766305.802883:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775766305.802887:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766305.802893:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766305/real 1775766305]  req@ffff9f14a04d3100 x1862022114476800/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766330 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766305.802896:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0380 x1862022114477440/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775766305.802918:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766305.802920:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:0.0:1775766305.802922:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:2.0:1775766305.802926:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3100 x1862022114476800/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766330 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766305.802927:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3683
00000100:00080000:0.0:1775766305.802933:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766305.802940:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00080000:2.0:1775766305.802945:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000008:00100000:0.0:1775766305.802946:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00080000:2.0:1775766305.802952:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766305.802955:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114476800:192.168.202.110@tcp:250:kworker.0
00000100:00100000:0.0:1775766305.802962:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0e00 x1862022114477568/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775766305.802972:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:2.0:1775766305.802976:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114476928:192.168.202.110@tcp:38:kworker.0
00000100:00080000:0.0:1775766305.802979:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775766305.802982:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766305.802988:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3683
00000100:00080000:0.0:1775766305.802994:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766305.803000:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000800:00000100:2.0:1775766305.803000:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000008:00100000:0.0:1775766305.803009:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775766305.803012:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766305/real 1775766305]  req@ffff9f14a04d2a00 x1862022114476928/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766330 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766305.803024:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d2300 x1862022114477696/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775766305.803025:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d2a00 x1862022114476928/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766330 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766305.803075:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766305.803079:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766305.803091:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114476928:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766305.803096:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114477056:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775766305.803112:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766305.803119:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766305/real 1775766305]  req@ffff9f14a04d3b80 x1862022114477056/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766330 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766305.803126:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3b80 x1862022114477056/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766330 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766305.803132:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766305.803134:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766305.803136:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114477056:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766305.803139:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114477184:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775766305.803149:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766305.803154:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766305/real 1775766305]  req@ffff9f14a04d0a80 x1862022114477184/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766330 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766305.803161:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0a80 x1862022114477184/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766330 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766305.803166:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766305.803168:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766305.803170:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114477184:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775766305.803173:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114477312:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775766305.803181:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766305.803186:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766305/real 1775766305]  req@ffff9f14a04d1f80 x1862022114477312/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766330 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766305.803193:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1f80 x1862022114477312/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766330 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766305.803198:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766305.803200:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766305.803201:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114477312:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775766305.803205:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114477440:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775766305.803213:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766305.803219:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766305/real 1775766305]  req@ffff9f14a04d0380 x1862022114477440/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766330 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766305.803225:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0380 x1862022114477440/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766330 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766305.803234:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766305.803236:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766305.803238:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114477440:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766305.803241:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114477568:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775766305.803250:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766305.803255:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766305/real 1775766305]  req@ffff9f14a04d0e00 x1862022114477568/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766330 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766305.803261:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0e00 x1862022114477568/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766330 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766305.803265:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766305.803267:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766305.803269:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114477568:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766305.803284:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114477696:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775766305.803293:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766305.803299:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766305/real 1775766305]  req@ffff9f14a04d2300 x1862022114477696/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766330 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766305.803306:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d2300 x1862022114477696/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766330 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766305.803310:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766305.803312:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766305.803314:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114477696:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775766306.058736:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766306.826657:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766307.082753:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766307.850650:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766308.106679:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766308.874667:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766309.130767:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766309.898617:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775766310.154779:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1775766310.922667:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766310.922676:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1775766310.922680:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775766310.922691:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:2.0:1775766310.922691:0:8:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3688
00000100:00080000:2.0:1775766310.922702:0:8:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766310.922710:0:8:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775766310.922750:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1c00 x1862022114477824/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766310.922778:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766310.922782:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775766310.922791:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766310.922795:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3688
00000100:00080000:2.0:1775766310.922799:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766310.922803:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766310.922812:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766310.922823:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3480 x1862022114477952/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775766310.922830:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114477824:192.168.202.110@tcp:250:kworker.0
00000100:00080000:2.0:1775766310.922850:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766310.922853:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775766310.922854:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766310.922859:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3688
00000100:00080000:2.0:1775766310.922862:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766310.922865:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766310.922869:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766310.922881:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2300 x1862022114478080/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766310.922888:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766310.922892:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775766310.922894:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766310.922896:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3688
00000100:00080000:2.0:1775766310.922899:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766310.922901:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766310.922904:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775766310.922912:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3100 x1862022114478208/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:3.0:1775766310.922916:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:2.0:1775766310.922926:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766310.922929:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1775766310.922931:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766310.922935:0:8:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3688
00000100:00080000:2.0:1775766310.922939:0:8:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766310.922942:0:8:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766310.922946:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775766310.922955:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1880 x1862022114478336/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766310.922962:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766310.922967:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00100000:3.0:1775766310.922967:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766310/real 1775766310]  req@ffff9f14852d1c00 x1862022114477824/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766335 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766310.922971:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766310.922981:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3688
00000100:00080000:2.0:1775766310.922987:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766310.922992:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766310.923013:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766310.923024:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0e00 x1862022114478464/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775766310.923032:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1c00 x1862022114477824/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766335 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766310.923035:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766310.923043:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775766310.923047:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766310.923057:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3688
00000100:00080000:2.0:1775766310.923062:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766310.923063:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766310.923066:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766310.923074:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00080000:3.0:1775766310.923077:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766310.923085:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114477824:192.168.202.110@tcp:250:kworker.0
00000100:00100000:2.0:1775766310.923091:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3b80 x1862022114478592/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766310.923105:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766310.923108:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775766310.923109:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766310.923113:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3688
00000100:00080000:2.0:1775766310.923115:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766310.923121:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775766310.923122:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114477952:192.168.202.110@tcp:38:kworker.0
00000008:00100000:2.0:1775766310.923125:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775766310.923137:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0000 x1862022114478720/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:3.0:1775766310.923167:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766310.923186:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766310/real 1775766310]  req@ffff9f14852d3480 x1862022114477952/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766335 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766310.923200:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3480 x1862022114477952/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766335 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766310.923207:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766310.923210:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766310.923214:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114477952:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766310.923221:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114478080:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775766310.923246:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766310.923260:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766310/real 1775766310]  req@ffff9f14852d2300 x1862022114478080/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766335 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766310.923277:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2300 x1862022114478080/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766335 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766310.923288:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766310.923292:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766310.923296:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114478080:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766310.923305:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114478208:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775766310.923335:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766310.923346:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766310/real 1775766310]  req@ffff9f14852d3100 x1862022114478208/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766335 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766310.923355:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3100 x1862022114478208/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766335 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766310.923362:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766310.923365:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766310.923367:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114478208:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775766310.923372:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114478336:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775766310.923385:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766310.923392:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766310/real 1775766310]  req@ffff9f14852d1880 x1862022114478336/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766335 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766310.923401:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1880 x1862022114478336/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766335 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766310.923407:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766310.923409:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766310.923411:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114478336:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775766310.923423:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114478464:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775766310.923436:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766310.923444:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766310/real 1775766310]  req@ffff9f14852d0e00 x1862022114478464/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766335 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766310.923456:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0e00 x1862022114478464/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766335 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766310.923466:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766310.923469:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766310.923473:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114478464:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766310.923480:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114478592:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775766310.923505:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766310.923518:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766310/real 1775766310]  req@ffff9f14852d3b80 x1862022114478592/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766335 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766310.923533:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3b80 x1862022114478592/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766335 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766310.923544:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766310.923548:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766310.923551:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114478592:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766310.923618:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114478720:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775766310.923645:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766310.923659:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766310/real 1775766310]  req@ffff9f14852d0000 x1862022114478720/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766335 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766310.923674:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0000 x1862022114478720/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766335 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766310.923685:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766310.923688:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766310.923691:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114478720:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775766311.178775:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766311.946682:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766312.202796:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766312.970714:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766313.226905:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766313.994666:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766314.250729:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766315.018771:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766315.274890:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766316.042654:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:1.0:1775766316.042757:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766316.042765:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1775766316.042769:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766316.042778:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3693
00000100:00080000:1.0:1775766316.042789:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766316.042796:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775766316.042838:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ffb80 x1862022114478848/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766316.042902:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766316.042907:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775766316.042910:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766316.042915:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3693
00000100:00080000:1.0:1775766316.042921:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766316.042927:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766316.042938:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775766316.042989:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdf80 x1862022114478976/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766316.043001:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766316.043005:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775766316.043007:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766316.043011:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3693
00000100:00080000:1.0:1775766316.043015:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766316.043019:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766316.043024:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766316.043032:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ffb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114478848:192.168.202.110@tcp:250:kworker.0
00000100:00100000:1.0:1775766316.043036:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff800 x1862022114479104/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766316.043050:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766316.043053:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775766316.043054:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766316.043057:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3693
00000100:00080000:1.0:1775766316.043061:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766316.043064:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766316.043067:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775766316.043078:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fea00 x1862022114479232/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766316.043086:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766316.043088:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1775766316.043090:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766316.043094:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3693
00000100:00080000:1.0:1775766316.043106:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766316.043113:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000800:00000100:3.0:1775766316.043117:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000008:00100000:1.0:1775766316.043125:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775766316.043171:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fce00 x1862022114479360/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766316.043184:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766316.043189:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775766316.043192:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:3.0:1775766316.043194:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766316/real 1775766316]  req@ffff9f14ab4ffb80 x1862022114478848/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766341 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766316.043203:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3693
00000100:00080000:1.0:1775766316.043214:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766316.043222:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775766316.043222:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ffb80 x1862022114478848/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766341 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:1.0:1775766316.043231:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00080000:3.0:1775766316.043238:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766316.043246:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775766316.043248:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fca80 x1862022114479488/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775766316.043254:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ffb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114478848:192.168.202.110@tcp:250:kworker.0
00000100:00080000:1.0:1775766316.043269:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:3.0:1775766316.043272:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114478976:192.168.202.110@tcp:38:kworker.0
00000100:00080000:1.0:1775766316.043282:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775766316.043285:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766316.043290:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3693
00000100:00080000:1.0:1775766316.043294:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000800:00000100:3.0:1775766316.043297:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:1.0:1775766316.043302:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766316.043307:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766316.043314:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766316/real 1775766316]  req@ffff9f14ab4fdf80 x1862022114478976/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766341 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775766316.043320:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fed80 x1862022114479616/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766316.043340:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:3.0:1775766316.043342:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdf80 x1862022114478976/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766341 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766316.043344:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775766316.043347:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766316.043356:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3693
00000100:00080000:1.0:1775766316.043364:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766316.043364:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775766316.043368:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00080000:3.0:1775766316.043374:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000008:00100000:1.0:1775766316.043375:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766316.043381:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114478976:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775766316.043387:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdc00 x1862022114479744/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775766316.043409:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114479104:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775766316.043426:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766316.043434:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766316/real 1775766316]  req@ffff9f14ab4ff800 x1862022114479104/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766341 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766316.043443:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff800 x1862022114479104/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766341 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766316.043461:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766316.043464:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766316.043466:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114479104:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766316.043471:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114479232:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775766316.043482:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766316.043488:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766316/real 1775766316]  req@ffff9f14ab4fea00 x1862022114479232/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766341 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766316.043498:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fea00 x1862022114479232/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766341 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766316.043504:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766316.043506:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766316.043509:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114479232:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775766316.043513:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114479360:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775766316.043523:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766316.043530:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766316/real 1775766316]  req@ffff9f14ab4fce00 x1862022114479360/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766341 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766316.043538:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fce00 x1862022114479360/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766341 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766316.043543:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766316.043545:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766316.043548:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114479360:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775766316.043552:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114479488:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775766316.043563:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766316.043601:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766316/real 1775766316]  req@ffff9f14ab4fca80 x1862022114479488/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766341 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766316.043612:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fca80 x1862022114479488/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766341 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766316.043624:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766316.043626:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766316.043629:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114479488:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766316.043633:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114479616:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775766316.043666:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766316.043676:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766316/real 1775766316]  req@ffff9f14ab4fed80 x1862022114479616/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766341 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766316.043686:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fed80 x1862022114479616/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766341 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766316.043693:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766316.043696:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766316.043698:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114479616:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766316.043703:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114479744:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775766316.043717:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766316.043724:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766316/real 1775766316]  req@ffff9f14ab4fdc00 x1862022114479744/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766341 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766316.043733:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdc00 x1862022114479744/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766341 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766316.043740:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766316.043742:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766316.043745:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114479744:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775766316.298889:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766317.066661:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775766317.322887:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766318.090618:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766318.346785:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766319.114680:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766319.370868:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766320.138644:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766320.394794:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1775766321.162687:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766321.162698:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00100000:2.0:1775766321.162699:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:0.0:1775766321.162703:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766321.162714:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3698
00000100:00080000:0.0:1775766321.162727:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766321.162735:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775766321.162794:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d1180 x1862022114479872/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775766321.162819:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766321.162822:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775766321.162824:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766321.162827:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3698
00000100:00080000:0.0:1775766321.162830:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766321.162834:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775766321.162845:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775766321.162853:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3800 x1862022114480000/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775766321.162861:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766321.162864:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:0.0:1775766321.162865:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766321.162869:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3698
00000100:00080000:0.0:1775766321.162871:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766321.162873:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775766321.162876:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775766321.162885:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d1500 x1862022114480128/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775766321.162886:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114479872:192.168.202.110@tcp:250:kworker.0
00000100:00080000:0.0:1775766321.162913:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766321.162915:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775766321.162917:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766321.162920:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3698
00000100:00080000:0.0:1775766321.162922:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766321.162925:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775766321.162939:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775766321.162949:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0000 x1862022114480256/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775766321.162955:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766321.162958:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1775766321.162960:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766321.162962:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3698
00000100:00080000:0.0:1775766321.162965:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766321.162967:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775766321.162970:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775766321.162976:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3480 x1862022114480384/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775766321.162982:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766321.162985:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775766321.162985:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766321.162988:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3698
00000100:00080000:0.0:1775766321.162991:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766321.162996:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000800:00000100:3.0:1775766321.162997:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000008:00100000:0.0:1775766321.163002:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775766321.163011:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d2300 x1862022114480512/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775766321.163018:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766321.163021:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:0.0:1775766321.163022:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766321.163024:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3698
00000100:00080000:0.0:1775766321.163026:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766321.163029:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775766321.163032:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775766321.163041:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0e00 x1862022114480640/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775766321.163047:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766321/real 1775766321]  req@ffff9f14a04d1180 x1862022114479872/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766346 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766321.163048:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766321.163056:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775766321.163059:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766321.163067:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3698
00000100:00080000:0.0:1775766321.163069:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766321.163071:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775766321.163075:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766321.163077:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1180 x1862022114479872/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766346 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766321.163085:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0380 x1862022114480768/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766321.163088:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766321.163096:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766321.163100:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114479872:192.168.202.110@tcp:250:kworker.0
00000100:00100000:3.0:1775766321.163105:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114480000:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775766321.163124:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766321.163135:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766321/real 1775766321]  req@ffff9f14a04d3800 x1862022114480000/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766346 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766321.163144:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3800 x1862022114480000/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766346 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766321.163150:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766321.163152:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766321.163154:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114480000:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766321.163176:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114480128:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775766321.163189:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766321.163195:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766321/real 1775766321]  req@ffff9f14a04d1500 x1862022114480128/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766346 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766321.163215:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1500 x1862022114480128/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766346 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766321.163228:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766321.163230:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766321.163232:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114480128:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766321.163236:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114480256:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775766321.163248:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766321.163254:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766321/real 1775766321]  req@ffff9f14a04d0000 x1862022114480256/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766346 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766321.163262:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0000 x1862022114480256/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766346 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766321.163268:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766321.163270:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766321.163272:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114480256:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775766321.163275:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114480384:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775766321.163285:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766321.163290:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766321/real 1775766321]  req@ffff9f14a04d3480 x1862022114480384/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766346 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766321.163303:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3480 x1862022114480384/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766346 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766321.163308:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766321.163310:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766321.163311:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114480384:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775766321.163315:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114480512:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775766321.163332:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766321.163337:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766321/real 1775766321]  req@ffff9f14a04d2300 x1862022114480512/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766346 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766321.163345:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d2300 x1862022114480512/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766346 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766321.163350:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766321.163352:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766321.163353:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114480512:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766321.163357:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114480640:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775766321.163367:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766321.163372:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766321/real 1775766321]  req@ffff9f14a04d0e00 x1862022114480640/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766346 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766321.163380:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0e00 x1862022114480640/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766346 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766321.163390:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766321.163402:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766321.163404:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114480640:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766321.163407:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114480768:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775766321.163417:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766321.163422:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766321/real 1775766321]  req@ffff9f14a04d0380 x1862022114480768/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766346 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766321.163428:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0380 x1862022114480768/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766346 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766321.163433:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766321.163434:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766321.163436:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114480768:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775766321.418816:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766322.186805:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766322.442780:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766323.210635:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766323.466776:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766324.234720:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775766324.490756:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766325.258738:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766325.514697:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766326.282731:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775766326.282806:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766326.282815:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775766326.282820:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766326.282830:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3703
00000100:00080000:3.0:1775766326.282845:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766326.282897:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775766326.282967:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746680 x1862022114480896/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766326.283005:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766326.283010:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775766326.283012:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766326.283016:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3703
00000100:00080000:3.0:1775766326.283021:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766326.283025:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766326.283036:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766326.283051:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747b80 x1862022114481024/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766326.283063:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766326.283070:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00100000:2.0:1775766326.283073:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114480896:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775766326.283073:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766326.283084:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3703
00000100:00080000:3.0:1775766326.283087:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766326.283090:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766326.283095:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766326.283107:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744e00 x1862022114481152/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766326.283118:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766326.283121:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775766326.283124:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766326.283128:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3703
00000100:00080000:3.0:1775766326.283132:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766326.283136:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766326.283142:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766326.283152:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744380 x1862022114481280/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766326.283162:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766326.283165:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775766326.283167:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000800:00000100:2.0:1775766326.283175:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775766326.283180:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3703
00000100:00080000:3.0:1775766326.283186:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766326.283189:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766326.283206:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766326.283218:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746a00 x1862022114481408/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766326.283226:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766326.283230:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775766326.283231:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766326.283234:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3703
00000100:00080000:3.0:1775766326.283238:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766326.283243:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766326.283248:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766326.283256:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766326/real 1775766326]  req@ffff9f1485746680 x1862022114480896/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766351 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766326.283262:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746300 x1862022114481536/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766326.283296:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766326.283299:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775766326.283302:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:2.0:1775766326.283308:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746680 x1862022114480896/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766351 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766326.283312:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3703
00000100:00080000:3.0:1775766326.283320:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766326.283330:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766326.283338:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00080000:2.0:1775766326.283339:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766326.283350:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766326.283355:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747480 x1862022114481664/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775766326.283361:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114480896:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775766326.283380:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766326.283383:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775766326.283396:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766326.283400:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3703
00000100:00080000:3.0:1775766326.283404:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766326.283413:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775766326.283414:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114481024:192.168.202.110@tcp:38:kworker.0
00000008:00100000:3.0:1775766326.283418:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766326.283433:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745880 x1862022114481792/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:2.0:1775766326.283469:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766326.283493:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766326/real 1775766326]  req@ffff9f1485747b80 x1862022114481024/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766351 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766326.283526:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747b80 x1862022114481024/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766351 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766326.283535:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766326.283539:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766326.283542:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114481024:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766326.283549:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114481152:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775766326.283607:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766326.283620:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766326/real 1775766326]  req@ffff9f1485744e00 x1862022114481152/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766351 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766326.283652:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744e00 x1862022114481152/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766351 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766326.283675:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766326.283680:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766326.283685:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114481152:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766326.283696:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114481280:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775766326.283732:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766326.283746:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766326/real 1775766326]  req@ffff9f1485744380 x1862022114481280/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766351 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766326.283758:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744380 x1862022114481280/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766351 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766326.283799:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766326.283802:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766326.283806:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114481280:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775766326.283813:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114481408:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775766326.283836:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766326.283847:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766326/real 1775766326]  req@ffff9f1485746a00 x1862022114481408/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766351 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766326.283898:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746a00 x1862022114481408/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766351 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766326.283907:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766326.283910:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766326.283914:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114481408:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775766326.283920:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114481536:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775766326.283942:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766326.283952:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766326/real 1775766326]  req@ffff9f1485746300 x1862022114481536/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766351 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766326.283963:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746300 x1862022114481536/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766351 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766326.283971:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766326.283974:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766326.283977:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114481536:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766326.283982:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114481664:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775766326.283996:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766326.284004:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766326/real 1775766326]  req@ffff9f1485747480 x1862022114481664/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766351 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766326.284015:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747480 x1862022114481664/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766351 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766326.284023:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766326.284025:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766326.284028:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114481664:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766326.284049:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114481792:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775766326.284064:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766326.284073:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766326/real 1775766326]  req@ffff9f1485745880 x1862022114481792/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766351 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766326.284085:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745880 x1862022114481792/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766351 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766326.284093:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766326.284095:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766326.284098:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114481792:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775766326.538705:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766327.306638:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766327.562801:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766328.330674:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766328.586717:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766329.354626:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766329.610704:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766330.378687:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766330.634716:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766331.402697:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:2.0:1775766331.402697:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766331.402725:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1775766331.402731:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766331.402742:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3708
00000100:00080000:2.0:1775766331.402755:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766331.402765:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775766331.402825:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1f80 x1862022114481920/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766331.402868:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766331.402872:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775766331.402874:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766331.402878:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3708
00000100:00080000:2.0:1775766331.402882:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766331.402886:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766331.402896:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766331.402910:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2a00 x1862022114482048/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775766331.402916:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114481920:192.168.202.110@tcp:250:kworker.0
00000100:00080000:2.0:1775766331.402929:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766331.402932:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775766331.402934:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766331.402938:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3708
00000100:00080000:2.0:1775766331.402940:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766331.402943:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766331.402947:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766331.402958:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1180 x1862022114482176/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766331.402975:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766331.402978:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775766331.402980:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766331.402983:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3708
00000100:00080000:2.0:1775766331.402986:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766331.402990:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766331.402994:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775766331.403004:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2680 x1862022114482304/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766331.403012:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766331.403015:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1775766331.403017:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766331.403020:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3708
00000100:00080000:2.0:1775766331.403024:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766331.403027:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766331.403032:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766331.403041:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114482048:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766331.403043:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3800 x1862022114482432/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766331.403075:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766331.403078:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775766331.403080:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766331.403086:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3708
00000100:00080000:2.0:1775766331.403094:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766331.403098:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766331.403104:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766331.403110:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114482176:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766331.403118:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0700 x1862022114482560/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766331.403134:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766331.403137:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775766331.403139:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766331.403146:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3708
00000100:00100000:3.0:1775766331.403146:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114482304:192.168.202.110@tcp:8:kworker.0
00000100:00080000:2.0:1775766331.403150:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766331.403158:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766331.403185:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766331.403192:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114482432:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775766331.403216:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2d80 x1862022114482688/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766331.403231:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766331.403235:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775766331.403237:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766331.403243:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3708
00000100:00080000:2.0:1775766331.403249:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:3.0:1775766331.403258:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114482560:192.168.202.110@tcp:38:kworker.0
00000100:00080000:2.0:1775766331.403280:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766331.403287:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775766331.403299:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0380 x1862022114482816/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775766331.403340:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114482688:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766331.403359:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114482816:192.168.202.110@tcp:8:kworker.0
00010000:00010000:3.0:1775766331.658743:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766332.426650:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766332.682752:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766333.450648:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766333.706705:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766334.474652:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000400:00000100:1.0:1775766334.474797:0:2393:0:(lib-socket.c:454:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.202.110:988
00000400:00000100:1.0:1775766334.474844:0:2393:0:(acceptor.c:111:lnet_connect_console_error()) Connection to 192.168.202.110@tcp at host 192.168.202.110 was unreachable: the network or that node may be down, or Lustre may be misconfigured.
00000800:00000100:1.0:1775766334.474860:0:2393:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:1.0:1775766334.474902:0:2393:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:1.0:1775766334.474922:0:2393:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:1.0:1775766334.474936:0:2393:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:1.0:1775766334.474943:0:2393:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:1.0:1775766334.474950:0:2393:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:1.0:1775766334.474958:0:2393:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:1.0:1775766334.474965:0:2393:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000100:00100000:3.0:1775766334.475063:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766331/real 1775766334]  req@ffff9f14852d2d80 x1862022114482688/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766356 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766334.475103:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2d80 x1862022114482688/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766356 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766334.475117:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766334.475123:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766334.475128:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114482688:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766334.475137:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766331/real 1775766334]  req@ffff9f14852d0380 x1862022114482816/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766356 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766334.475168:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0380 x1862022114482816/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766356 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766334.475177:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766334.475182:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766334.475185:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114482816:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775766334.475195:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766331/real 1775766334]  req@ffff9f14852d3800 x1862022114482432/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766356 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766334.475217:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3800 x1862022114482432/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766356 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766334.475225:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766334.475229:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766334.475232:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114482432:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775766334.475239:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766331/real 1775766334]  req@ffff9f14852d0700 x1862022114482560/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766356 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766334.475252:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0700 x1862022114482560/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766356 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766334.475260:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766334.475264:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766334.475268:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114482560:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766334.475275:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766331/real 1775766334]  req@ffff9f14852d2a00 x1862022114482048/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766356 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766334.475286:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2a00 x1862022114482048/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766356 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766334.475294:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766334.475298:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766334.475301:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114482048:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766334.475308:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766331/real 1775766334]  req@ffff9f14852d1180 x1862022114482176/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766356 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766334.475321:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1180 x1862022114482176/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766356 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766334.475329:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766334.475333:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766334.475335:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114482176:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766334.475342:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766331/real 1775766334]  req@ffff9f14852d2680 x1862022114482304/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766356 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766334.475355:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2680 x1862022114482304/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766356 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766334.475364:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766334.475368:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766334.475370:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114482304:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775766334.475379:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766331/real 1775766334]  req@ffff9f14852d1f80 x1862022114481920/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766356 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766334.475391:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1f80 x1862022114481920/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766356 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766334.475399:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766334.475403:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766334.475406:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114481920:192.168.202.110@tcp:250:kworker.0
00010000:00010000:2.0:1775766334.730754:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766335.498661:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766335.754818:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1775766336.522725:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1775766336.522731:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:2.0:1775766336.522734:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1775766336.522746:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766336.522783:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3714
00000100:00080000:2.0:1775766336.522796:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766336.522804:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775766336.522840:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0a80 x1862022114482944/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766336.522878:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766336.522883:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775766336.522886:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766336.522892:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3714
00000100:00080000:2.0:1775766336.522897:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766336.522902:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766336.522913:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766336.522931:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1500 x1862022114483072/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775766336.522946:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114482944:192.168.202.110@tcp:250:kworker.0
00000100:00080000:2.0:1775766336.522948:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766336.522960:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775766336.522962:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766336.522968:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3714
00000100:00080000:2.0:1775766336.522973:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766336.522978:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766336.522984:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766336.523003:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1f80 x1862022114483200/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766336.523013:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766336.523017:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775766336.523020:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766336.523024:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3714
00000100:00080000:2.0:1775766336.523029:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766336.523032:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766336.523039:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775766336.523054:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2680 x1862022114483328/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1775766336.523067:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:2.0:1775766336.523073:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766336.523078:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1775766336.523080:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766336.523085:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3714
00000100:00080000:2.0:1775766336.523090:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766336.523094:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766336.523101:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775766336.523117:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1180 x1862022114483456/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766336.523128:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766336.523132:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775766336.523135:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766336.523140:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3714
00000100:00080000:2.0:1775766336.523145:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766336.523150:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766336.523156:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775766336.523159:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766336/real 1775766336]  req@ffff9f14852d0a80 x1862022114482944/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766361 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766336.523175:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2a00 x1862022114483584/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766336.523186:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766336.523189:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775766336.523193:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766336.523198:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3714
00000100:00080000:2.0:1775766336.523203:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766336.523207:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775766336.523212:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0a80 x1862022114482944/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766361 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:2.0:1775766336.523218:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00080000:0.0:1775766336.523244:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00100000:2.0:1775766336.523251:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0700 x1862022114483712/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775766336.523258:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766336.523276:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114482944:192.168.202.110@tcp:250:kworker.0
00000100:00080000:2.0:1775766336.523299:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766336.523304:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775766336.523307:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766336.523316:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3714
00000100:00080000:2.0:1775766336.523323:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766336.523327:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766336.523337:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775766336.523340:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114483072:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766336.523355:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3800 x1862022114483840/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1775766336.523393:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766336.523417:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766336/real 1775766336]  req@ffff9f14852d1500 x1862022114483072/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766361 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766336.523436:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1500 x1862022114483072/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766361 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766336.523452:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766336.523459:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766336.523464:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114483072:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766336.523474:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114483200:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766336.523503:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766336.523522:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766336/real 1775766336]  req@ffff9f14852d1f80 x1862022114483200/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766361 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766336.523586:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1f80 x1862022114483200/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766361 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766336.523597:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766336.523603:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766336.523609:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114483200:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766336.523618:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114483328:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766336.523645:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766336.523656:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766336/real 1775766336]  req@ffff9f14852d2680 x1862022114483328/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766361 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766336.523668:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2680 x1862022114483328/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766361 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766336.523676:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766336.523680:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766336.523683:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114483328:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775766336.523689:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114483456:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766336.523708:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766336.523725:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766336/real 1775766336]  req@ffff9f14852d1180 x1862022114483456/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766361 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766336.523739:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1180 x1862022114483456/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766361 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766336.523771:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766336.523775:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766336.523779:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114483456:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775766336.523786:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114483584:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766336.523810:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766336.523823:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766336/real 1775766336]  req@ffff9f14852d2a00 x1862022114483584/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766361 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766336.523839:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2a00 x1862022114483584/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766361 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766336.523849:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766336.523855:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766336.523858:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114483584:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766336.523865:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114483712:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766336.523887:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766336.523915:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766336/real 1775766336]  req@ffff9f14852d0700 x1862022114483712/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766361 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766336.523927:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0700 x1862022114483712/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766361 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766336.523936:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766336.523940:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766336.523944:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114483712:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766336.523987:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114483840:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766336.524018:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766336.524031:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766336/real 1775766336]  req@ffff9f14852d3800 x1862022114483840/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766361 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766336.524044:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3800 x1862022114483840/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766361 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766336.524053:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766336.524056:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766336.524060:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114483840:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775766336.778838:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766337.546688:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766337.802763:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766338.570627:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775766338.826767:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766339.594625:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766339.850739:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766340.618649:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766340.874747:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1775766341.642683:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766341.642694:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1775766341.642699:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766341.642723:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3719
00000100:00100000:0.0:1775766341.642726:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:1.0:1775766341.642738:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766341.642747:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775766341.642784:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fc380 x1862022114483968/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766341.642811:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766341.642815:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775766341.642816:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766341.642822:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3719
00000100:00080000:1.0:1775766341.642828:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766341.642834:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766341.642845:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775766341.642851:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114483968:192.168.202.110@tcp:250:kworker.0
00000100:00100000:1.0:1775766341.642866:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fe300 x1862022114484096/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766341.642879:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766341.642884:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775766341.642885:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766341.642890:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3719
00000100:00080000:1.0:1775766341.642894:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766341.642898:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766341.642904:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775766341.642928:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fd880 x1862022114484224/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766341.642938:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766341.642941:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775766341.642943:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000800:00000100:0.0:1775766341.642950:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:1.0:1775766341.642957:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3719
00000100:00080000:1.0:1775766341.642964:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766341.642971:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766341.642979:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775766341.642998:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff100 x1862022114484352/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775766341.643006:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766341/real 1775766341]  req@ffff9f14ab4fc380 x1862022114483968/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766366 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766341.643026:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766341.643028:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1775766341.643029:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766341.643040:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3719
00000100:00080000:1.0:1775766341.643050:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766341.643056:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766341.643062:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775766341.643076:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fc380 x1862022114483968/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766366 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775766341.643078:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdc00 x1862022114484480/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775766341.643108:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775766341.643113:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766341.643115:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766341.643126:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114483968:192.168.202.110@tcp:250:kworker.0
00000100:00080000:1.0:1775766341.643126:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775766341.643137:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766341.643143:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3719
00000100:00080000:1.0:1775766341.643148:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766341.643153:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766341.643160:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775766341.643163:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114484096:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775766341.643191:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fed80 x1862022114484608/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1775766341.643198:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766341.643215:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766341/real 1775766341]  req@ffff9f14ab4fe300 x1862022114484096/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766366 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766341.643215:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766341.643231:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775766341.643238:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766341.643258:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3719
00000100:00080000:1.0:1775766341.643263:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766341.643267:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775766341.643271:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fe300 x1862022114484096/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766366 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:1.0:1775766341.643278:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00080000:0.0:1775766341.643289:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766341.643293:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766341.643300:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114484096:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775766341.643300:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fca80 x1862022114484736/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775766341.643322:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114484224:192.168.202.110@tcp:38:kworker.0
00000100:00080000:1.0:1775766341.643326:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766341.643330:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775766341.643331:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766341.643336:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3719
00000100:00080000:1.0:1775766341.643342:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000800:00000100:0.0:1775766341.643346:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:1.0:1775766341.643350:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766341.643358:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775766341.643363:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766341/real 1775766341]  req@ffff9f14ab4fd880 x1862022114484224/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766366 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775766341.643372:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fce00 x1862022114484864/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775766341.643388:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fd880 x1862022114484224/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766366 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766341.643397:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766341.643400:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766341.643402:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114484224:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766341.643410:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114484352:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766341.643430:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766341.643443:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766341/real 1775766341]  req@ffff9f14ab4ff100 x1862022114484352/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766366 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766341.643456:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff100 x1862022114484352/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766366 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766341.643465:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766341.643468:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766341.643471:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114484352:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775766341.643487:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114484480:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766341.643511:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766341.643522:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766341/real 1775766341]  req@ffff9f14ab4fdc00 x1862022114484480/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766366 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766341.643530:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdc00 x1862022114484480/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766366 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766341.643536:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766341.643539:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766341.643541:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114484480:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775766341.643576:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114484608:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766341.643591:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766341.643598:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766341/real 1775766341]  req@ffff9f14ab4fed80 x1862022114484608/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766366 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766341.643606:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fed80 x1862022114484608/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766366 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766341.643613:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766341.643615:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766341.643617:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114484608:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766341.643621:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114484736:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766341.643632:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766341.643637:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766341/real 1775766341]  req@ffff9f14ab4fca80 x1862022114484736/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766366 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766341.643645:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fca80 x1862022114484736/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766366 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766341.643650:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766341.643652:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766341.643653:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114484736:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766341.643657:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114484864:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766341.643667:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766341.643673:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766341/real 1775766341]  req@ffff9f14ab4fce00 x1862022114484864/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766366 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766341.643680:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fce00 x1862022114484864/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766366 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766341.643685:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766341.643686:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766341.643689:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114484864:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775766341.898798:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766342.666677:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766342.922887:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766343.690627:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766343.946767:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766344.714761:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766344.970736:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766345.738657:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775766345.994797:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766346.762651:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:1.0:1775766346.762666:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766346.762680:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1775766346.762684:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766346.762695:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3724
00000100:00080000:1.0:1775766346.762708:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766346.762716:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775766346.762763:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fea00 x1862022114484992/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766346.762790:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766346.762793:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775766346.762794:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766346.762798:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3724
00000100:00080000:1.0:1775766346.762801:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766346.762804:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766346.762819:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775766346.762824:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114484992:192.168.202.110@tcp:250:kworker.0
00000100:00100000:1.0:1775766346.762832:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff800 x1862022114485120/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766346.762840:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766346.762843:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775766346.762844:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766346.762847:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3724
00000100:00080000:1.0:1775766346.762848:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766346.762851:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766346.762855:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775766346.762862:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdf80 x1862022114485248/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766346.762874:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766346.762876:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775766346.762878:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766346.762880:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3724
00000100:00080000:1.0:1775766346.762896:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766346.762899:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766346.762902:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:0.0:1775766346.762907:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775766346.762911:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ffb80 x1862022114485376/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766346.762918:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766346.762921:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1775766346.762923:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766346.762926:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3724
00000100:00080000:1.0:1775766346.762930:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766346.762932:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766346.762935:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775766346.762944:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fce00 x1862022114485504/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766346.762950:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766346.762955:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00100000:0.0:1775766346.762956:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766346/real 1775766346]  req@ffff9f14ab4fea00 x1862022114484992/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766371 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766346.762957:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766346.762965:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3724
00000100:00080000:1.0:1775766346.762970:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766346.762980:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775766346.762982:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fea00 x1862022114484992/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766371 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:1.0:1775766346.762987:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00080000:0.0:1775766346.762996:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766346.763002:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775766346.763002:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fca80 x1862022114485632/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775766346.763009:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114484992:192.168.202.110@tcp:250:kworker.0
00000100:00080000:1.0:1775766346.763020:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766346.763023:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775766346.763024:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766346.763027:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3724
00000100:00080000:1.0:1775766346.763030:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766346.763032:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766346.763036:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775766346.763039:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114485120:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775766346.763045:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fed80 x1862022114485760/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766346.763054:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766346.763057:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775766346.763059:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000800:00000100:0.0:1775766346.763063:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:1.0:1775766346.763067:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3724
00000100:00080000:1.0:1775766346.763071:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766346.763074:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766346.763078:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775766346.763080:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766346/real 1775766346]  req@ffff9f14ab4ff800 x1862022114485120/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766371 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775766346.763088:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdc00 x1862022114485888/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775766346.763092:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff800 x1862022114485120/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766371 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766346.763101:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766346.763107:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766346.763115:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114485120:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766346.763125:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114485248:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766346.763139:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766346.763146:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766346/real 1775766346]  req@ffff9f14ab4fdf80 x1862022114485248/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766371 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766346.763152:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdf80 x1862022114485248/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766371 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766346.763158:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766346.763160:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766346.763162:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114485248:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766346.763166:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ffb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114485376:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766346.763176:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766346.763181:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766346/real 1775766346]  req@ffff9f14ab4ffb80 x1862022114485376/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766371 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766346.763188:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ffb80 x1862022114485376/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766371 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766346.763194:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766346.763195:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766346.763197:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ffb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114485376:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775766346.763200:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114485504:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766346.763209:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766346.763215:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766346/real 1775766346]  req@ffff9f14ab4fce00 x1862022114485504/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766371 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766346.763223:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fce00 x1862022114485504/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766371 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766346.763228:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766346.763229:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766346.763231:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114485504:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775766346.763234:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114485632:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766346.763244:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766346.763249:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766346/real 1775766346]  req@ffff9f14ab4fca80 x1862022114485632/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766371 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766346.763256:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fca80 x1862022114485632/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766371 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766346.763278:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766346.763280:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766346.763282:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114485632:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766346.763296:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114485760:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766346.763308:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766346.763315:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766346/real 1775766346]  req@ffff9f14ab4fed80 x1862022114485760/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766371 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766346.763324:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fed80 x1862022114485760/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766371 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766346.763334:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766346.763337:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766346.763341:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114485760:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766346.763348:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114485888:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766346.763370:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766346.763382:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766346/real 1775766346]  req@ffff9f14ab4fdc00 x1862022114485888/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766371 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766346.763396:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdc00 x1862022114485888/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766371 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766346.763405:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766346.763408:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766346.763410:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114485888:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775766347.018754:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766347.786685:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766348.042780:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766348.810685:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766349.066686:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766349.834723:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766350.090822:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766350.858743:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766351.114806:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766351.882722:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:2.0:1775766351.882753:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766351.882762:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1775766351.882767:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766351.882777:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3729
00000100:00080000:2.0:1775766351.882787:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766351.882792:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775766351.882826:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0380 x1862022114486016/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766351.882853:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766351.882856:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775766351.882857:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766351.882860:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3729
00000100:00080000:2.0:1775766351.882863:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766351.882866:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766351.882876:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766351.882887:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2d80 x1862022114486144/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775766351.882893:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114486016:192.168.202.110@tcp:250:kworker.0
00000100:00080000:2.0:1775766351.882912:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766351.882915:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775766351.882916:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766351.882919:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3729
00000100:00080000:2.0:1775766351.882922:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766351.882926:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766351.882930:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766351.882939:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0000 x1862022114486272/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766351.882946:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766351.882948:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775766351.882950:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766351.882952:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3729
00000100:00080000:2.0:1775766351.882954:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766351.882957:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766351.882960:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775766351.882966:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3b80 x1862022114486400/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766351.882972:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766351.882974:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1775766351.882976:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766351.882978:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3729
00000100:00080000:2.0:1775766351.882981:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766351.882983:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766351.882986:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:0.0:1775766351.882991:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766351.883001:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0e00 x1862022114486528/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766351.883010:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766351.883013:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775766351.883016:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766351.883020:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3729
00000100:00080000:2.0:1775766351.883023:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766351.883027:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766351.883034:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775766351.883043:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766351/real 1775766351]  req@ffff9f14852d0380 x1862022114486016/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766376 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766351.883053:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1880 x1862022114486656/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775766351.883067:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0380 x1862022114486016/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766376 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766351.883069:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766351.883090:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775766351.883094:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766351.883096:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766351.883102:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3729
00000100:00080000:0.0:1775766351.883106:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:2.0:1775766351.883112:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1775766351.883118:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114486016:192.168.202.110@tcp:250:kworker.0
00000100:00080000:2.0:1775766351.883124:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766351.883134:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766351.883148:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3100 x1862022114486784/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775766351.883153:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114486144:192.168.202.110@tcp:38:kworker.0
00000100:00080000:2.0:1775766351.883176:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766351.883179:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775766351.883180:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766351.883186:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3729
00000100:00080000:2.0:1775766351.883193:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766351.883196:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766351.883201:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:0.0:1775766351.883204:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766351.883218:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2300 x1862022114486912/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775766351.883228:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766351/real 1775766351]  req@ffff9f14852d2d80 x1862022114486144/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766376 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766351.883243:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2d80 x1862022114486144/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766376 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766351.883249:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766351.883253:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766351.883255:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114486144:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766351.883263:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114486272:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766351.883277:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766351.883283:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766351/real 1775766351]  req@ffff9f14852d0000 x1862022114486272/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766376 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766351.883290:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0000 x1862022114486272/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766376 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766351.883296:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766351.883298:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766351.883300:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114486272:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766351.883305:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114486400:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766351.883316:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766351.883322:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766351/real 1775766351]  req@ffff9f14852d3b80 x1862022114486400/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766376 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766351.883329:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3b80 x1862022114486400/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766376 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766351.883334:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766351.883336:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766351.883338:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114486400:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775766351.883341:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114486528:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766351.883351:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766351.883356:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766351/real 1775766351]  req@ffff9f14852d0e00 x1862022114486528/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766376 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766351.883363:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0e00 x1862022114486528/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766376 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766351.883368:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766351.883369:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766351.883371:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114486528:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775766351.883375:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114486656:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766351.883385:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766351.883391:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766351/real 1775766351]  req@ffff9f14852d1880 x1862022114486656/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766376 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766351.883399:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1880 x1862022114486656/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766376 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766351.883405:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766351.883406:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766351.883408:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114486656:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766351.883422:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114486784:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766351.883432:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766351.883437:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766351/real 1775766351]  req@ffff9f14852d3100 x1862022114486784/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766376 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766351.883444:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3100 x1862022114486784/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766376 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766351.883448:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766351.883450:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766351.883452:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114486784:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766351.883455:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114486912:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766351.883465:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766351.883470:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766351/real 1775766351]  req@ffff9f14852d2300 x1862022114486912/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766376 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766351.883477:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2300 x1862022114486912/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766376 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766351.883488:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766351.883490:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766351.883492:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114486912:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775766352.138767:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766352.906714:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775766353.162788:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766353.930714:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766354.186811:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766354.954660:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766355.210715:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766355.978684:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766356.234760:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766357.002651:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:1.0:1775766357.002721:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766357.002729:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1775766357.002734:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766357.002745:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3734
00000100:00080000:1.0:1775766357.002758:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766357.002765:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775766357.002829:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff100 x1862022114487040/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766357.002857:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766357.002861:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775766357.002862:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766357.002867:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3734
00000100:00080000:1.0:1775766357.002879:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766357.002884:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766357.002895:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775766357.002896:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114487040:192.168.202.110@tcp:250:kworker.0
00000100:00100000:1.0:1775766357.002931:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fd880 x1862022114487168/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766357.002942:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766357.002945:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775766357.002947:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766357.002952:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3734
00000100:00080000:1.0:1775766357.002969:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766357.002973:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000800:00000100:0.0:1775766357.002980:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000008:00100000:1.0:1775766357.002982:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775766357.002995:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fe300 x1862022114487296/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766357.003004:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766357.003008:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775766357.003010:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766357.003016:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3734
00000100:00080000:1.0:1775766357.003021:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766357.003030:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775766357.003032:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766357/real 1775766357]  req@ffff9f14ab4ff100 x1862022114487040/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766382 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:1.0:1775766357.003037:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775766357.003058:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fc380 x1862022114487424/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775766357.003065:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff100 x1862022114487040/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766382 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766357.003087:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766357.003095:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1775766357.003097:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766357.003101:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775766357.003102:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3734
00000100:00080000:1.0:1775766357.003108:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766357.003114:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00080000:0.0:1775766357.003120:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000008:00100000:1.0:1775766357.003120:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775766357.003129:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114487040:192.168.202.110@tcp:250:kworker.0
00000100:00100000:1.0:1775766357.003134:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdc00 x1862022114487552/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766357.003142:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766357.003145:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775766357.003147:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766357.003153:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3734
00000100:00100000:0.0:1775766357.003157:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114487168:192.168.202.110@tcp:38:kworker.0
00000100:00080000:1.0:1775766357.003158:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766357.003170:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766357.003174:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775766357.003188:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fed80 x1862022114487680/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766357.003199:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000800:00000100:0.0:1775766357.003201:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:1.0:1775766357.003210:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775766357.003213:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766357.003219:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3734
00000100:00080000:1.0:1775766357.003223:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766357.003230:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775766357.003231:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766357/real 1775766357]  req@ffff9f14ab4fd880 x1862022114487168/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766382 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:1.0:1775766357.003241:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775766357.003254:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fd880 x1862022114487168/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766382 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775766357.003258:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fca80 x1862022114487808/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775766357.003307:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775766357.003319:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766357.003322:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00080000:1.0:1775766357.003332:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00100000:0.0:1775766357.003337:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114487168:192.168.202.110@tcp:38:kworker.0
00000100:00080000:1.0:1775766357.003337:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766357.003352:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3734
00000100:00080000:1.0:1775766357.003356:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1775766357.003360:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114487296:192.168.202.110@tcp:38:kworker.0
00000100:00080000:1.0:1775766357.003369:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766357.003376:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:0.0:1775766357.003402:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775766357.003409:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fce00 x1862022114487936/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775766357.003415:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766357/real 1775766357]  req@ffff9f14ab4fe300 x1862022114487296/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766382 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766357.003455:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fe300 x1862022114487296/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766382 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766357.003462:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766357.003466:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766357.003468:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114487296:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766357.003474:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114487424:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766357.003489:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766357.003496:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766357/real 1775766357]  req@ffff9f14ab4fc380 x1862022114487424/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766382 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766357.003503:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fc380 x1862022114487424/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766382 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766357.003515:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766357.003517:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766357.003520:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114487424:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775766357.003524:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114487552:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766357.003536:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766357.003542:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766357/real 1775766357]  req@ffff9f14ab4fdc00 x1862022114487552/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766382 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766357.003550:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdc00 x1862022114487552/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766382 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766357.003588:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766357.003591:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766357.003593:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114487552:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775766357.003620:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114487680:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766357.003636:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766357.003644:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766357/real 1775766357]  req@ffff9f14ab4fed80 x1862022114487680/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766382 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766357.003652:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fed80 x1862022114487680/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766382 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766357.003657:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766357.003660:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766357.003662:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114487680:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766357.003666:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114487808:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766357.003676:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766357.003682:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766357/real 1775766357]  req@ffff9f14ab4fca80 x1862022114487808/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766382 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766357.003689:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fca80 x1862022114487808/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766382 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766357.003694:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766357.003696:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766357.003698:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114487808:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766357.003702:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114487936:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766357.003712:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766357.003717:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766357/real 1775766357]  req@ffff9f14ab4fce00 x1862022114487936/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766382 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766357.003725:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fce00 x1862022114487936/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766382 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766357.003730:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766357.003732:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766357.003734:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114487936:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775766357.258762:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766358.026695:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766358.282788:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766359.050624:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766359.306725:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766360.074642:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775766360.330771:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766361.098701:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766361.354777:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1775766362.122700:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1775766362.122711:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:1.0:1775766362.122711:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1775766362.122716:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766362.122731:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3739
00000100:00080000:1.0:1775766362.122743:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766362.122750:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775766362.122799:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ffb80 x1862022114488064/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766362.122828:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766362.122832:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775766362.122834:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766362.122838:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3739
00000100:00080000:1.0:1775766362.122841:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766362.122844:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766362.122856:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775766362.122872:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ffb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114488064:192.168.202.110@tcp:250:kworker.0
00000100:00100000:1.0:1775766362.122872:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdf80 x1862022114488192/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766362.122904:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766362.122908:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775766362.122910:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766362.122915:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3739
00000100:00080000:1.0:1775766362.122918:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766362.122923:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766362.122928:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775766362.122938:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff800 x1862022114488320/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766362.122947:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766362.122951:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775766362.122955:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000800:00000100:0.0:1775766362.122957:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:1.0:1775766362.122962:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3739
00000100:00080000:1.0:1775766362.122965:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766362.122969:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766362.122975:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775766362.122986:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fea00 x1862022114488448/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766362.122995:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1775766362.123002:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766362/real 1775766362]  req@ffff9f14ab4ffb80 x1862022114488064/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766387 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766362.123003:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1775766362.123009:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766362.123023:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3739
00000100:00080000:1.0:1775766362.123034:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766362.123039:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775766362.123046:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ffb80 x1862022114488064/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766387 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:1.0:1775766362.123061:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00080000:0.0:1775766362.123077:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766362.123084:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775766362.123084:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fce00 x1862022114488576/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775766362.123099:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ffb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114488064:192.168.202.110@tcp:250:kworker.0
00000100:00080000:1.0:1775766362.123121:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766362.123125:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775766362.123127:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766362.123131:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3739
00000100:00080000:1.0:1775766362.123135:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1775766362.123140:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114488192:192.168.202.110@tcp:38:kworker.0
00000100:00080000:1.0:1775766362.123146:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766362.123150:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775766362.123159:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fca80 x1862022114488704/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1775766362.123168:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:1.0:1775766362.123174:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766362.123177:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775766362.123178:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766362.123184:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3739
00000100:00080000:1.0:1775766362.123187:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766362.123191:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766362.123195:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775766362.123196:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766362/real 1775766362]  req@ffff9f14ab4fdf80 x1862022114488192/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766387 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775766362.123207:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fed80 x1862022114488832/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766362.123233:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766362.123236:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775766362.123240:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775766362.123241:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdf80 x1862022114488192/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766387 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766362.123256:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3739
00000100:00080000:1.0:1775766362.123266:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766362.123270:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775766362.123271:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766362.123281:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00080000:0.0:1775766362.123282:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766362.123289:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114488192:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775766362.123295:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdc00 x1862022114488960/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775766362.123300:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114488320:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766362.123354:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766362.123367:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766362/real 1775766362]  req@ffff9f14ab4ff800 x1862022114488320/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766387 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766362.123379:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff800 x1862022114488320/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766387 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766362.123390:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766362.123393:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766362.123395:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114488320:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766362.123399:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114488448:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766362.123410:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766362.123416:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766362/real 1775766362]  req@ffff9f14ab4fea00 x1862022114488448/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766387 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766362.123423:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fea00 x1862022114488448/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766387 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766362.123431:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766362.123434:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766362.123437:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114488448:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775766362.123444:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114488576:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766362.123480:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766362.123493:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766362/real 1775766362]  req@ffff9f14ab4fce00 x1862022114488576/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766387 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766362.123520:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fce00 x1862022114488576/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766387 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766362.123527:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766362.123530:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766362.123533:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114488576:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775766362.123595:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114488704:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766362.123612:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766362.123620:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766362/real 1775766362]  req@ffff9f14ab4fca80 x1862022114488704/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766387 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766362.123628:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fca80 x1862022114488704/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766387 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766362.123633:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766362.123635:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766362.123638:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114488704:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766362.123642:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114488832:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766362.123651:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766362.123657:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766362/real 1775766362]  req@ffff9f14ab4fed80 x1862022114488832/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766387 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766362.123664:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fed80 x1862022114488832/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766387 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766362.123669:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766362.123670:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766362.123672:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114488832:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766362.123676:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114488960:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766362.123684:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766362.123690:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766362/real 1775766362]  req@ffff9f14ab4fdc00 x1862022114488960/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766387 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766362.123697:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdc00 x1862022114488960/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766387 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766362.123702:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766362.123704:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766362.123706:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114488960:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775766362.378735:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766363.146652:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766363.402655:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766364.170773:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766364.426706:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766365.194671:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766365.450711:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766366.218644:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766366.474671:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766367.242711:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:1.0:1775766367.242722:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766367.242733:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1775766367.242739:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766367.242751:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3744
00000100:00080000:1.0:1775766367.242763:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766367.242775:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775766367.242827:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fc380 x1862022114489088/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766367.242857:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766367.242862:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775766367.242864:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766367.242869:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3744
00000100:00080000:1.0:1775766367.242873:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766367.242877:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766367.242888:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775766367.242906:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fe300 x1862022114489216/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766367.242919:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766367.242924:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775766367.242926:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766367.242942:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3744
00000100:00080000:1.0:1775766367.242946:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766367.242958:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775766367.242959:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114489088:192.168.202.110@tcp:250:kworker.0
00000008:00100000:1.0:1775766367.242972:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775766367.242997:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fd880 x1862022114489344/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766367.243011:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766367.243016:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775766367.243019:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766367.243025:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3744
00000100:00080000:1.0:1775766367.243069:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000800:00000100:0.0:1775766367.243079:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:1.0:1775766367.243084:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766367.243092:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775766367.243110:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff100 x1862022114489472/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766367.243123:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766367.243127:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1775766367.243130:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775766367.243134:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766367/real 1775766367]  req@ffff9f14ab4fc380 x1862022114489088/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766392 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766367.243140:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3744
00000100:00080000:1.0:1775766367.243154:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766367.243161:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775766367.243168:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fc380 x1862022114489088/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766392 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:1.0:1775766367.243168:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00080000:0.0:1775766367.243186:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766367.243193:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775766367.243197:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdc00 x1862022114489600/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775766367.243200:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114489088:192.168.202.110@tcp:250:kworker.0
00000100:00100000:0.0:1775766367.243216:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114489216:192.168.202.110@tcp:38:kworker.0
00000100:00080000:1.0:1775766367.243240:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766367.243244:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775766367.243246:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000800:00000100:0.0:1775766367.243252:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:1.0:1775766367.243258:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3744
00000100:00080000:1.0:1775766367.243265:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766367.243270:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766367.243277:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775766367.243279:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766367/real 1775766367]  req@ffff9f14ab4fe300 x1862022114489216/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766392 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775766367.243295:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fed80 x1862022114489728/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775766367.243302:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fe300 x1862022114489216/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766392 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766367.243339:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775766367.243343:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766367.243348:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766367.243361:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114489216:192.168.202.110@tcp:38:kworker.0
00000100:00080000:1.0:1775766367.243363:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775766367.243373:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766367.243380:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3744
00000100:00080000:1.0:1775766367.243384:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766367.243400:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766367.243406:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775766367.243424:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fca80 x1862022114489856/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775766367.243436:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114489344:192.168.202.110@tcp:38:kworker.0
00000100:00080000:1.0:1775766367.243441:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766367.243454:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775766367.243490:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766367.243499:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3744
00000100:00080000:1.0:1775766367.243517:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000800:00000100:0.0:1775766367.243522:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:1.0:1775766367.243528:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766367.243535:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775766367.243537:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766367/real 1775766367]  req@ffff9f14ab4fd880 x1862022114489344/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766392 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766367.243555:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fd880 x1862022114489344/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766392 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766367.243588:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766367.243592:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766367.243596:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114489344:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775766367.243603:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fce00 x1862022114489984/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775766367.243611:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114489472:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766367.243653:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766367.243669:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766367/real 1775766367]  req@ffff9f14ab4ff100 x1862022114489472/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766392 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766367.243679:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff100 x1862022114489472/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766392 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766367.243685:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766367.243688:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766367.243690:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114489472:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775766367.243695:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114489600:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766367.243711:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766367.243717:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766367/real 1775766367]  req@ffff9f14ab4fdc00 x1862022114489600/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766392 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766367.243725:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdc00 x1862022114489600/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766392 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766367.243730:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766367.243732:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766367.243735:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114489600:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775766367.243739:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114489728:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766367.243750:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766367.243755:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766367/real 1775766367]  req@ffff9f14ab4fed80 x1862022114489728/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766392 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766367.243762:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fed80 x1862022114489728/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766392 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766367.243768:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766367.243769:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766367.243772:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114489728:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766367.243787:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114489856:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766367.243832:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766367.243851:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766367/real 1775766367]  req@ffff9f14ab4fca80 x1862022114489856/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766392 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766367.243874:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fca80 x1862022114489856/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766392 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766367.243890:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766367.243896:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766367.243902:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114489856:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766367.243913:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114489984:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766367.243942:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766367.243956:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766367/real 1775766367]  req@ffff9f14ab4fce00 x1862022114489984/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766392 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766367.243970:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fce00 x1862022114489984/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766392 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766367.243979:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766367.243983:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766367.243986:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114489984:192.168.202.110@tcp:8:kworker.0
00010000:00010000:3.0:1775766367.498687:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766368.266635:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766368.522668:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766369.290712:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766369.546842:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766370.314633:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766370.570723:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766371.338698:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766371.594809:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775766372.362668:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:2.0:1775766372.362721:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766372.362729:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1775766372.362735:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766372.362745:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3749
00000100:00080000:2.0:1775766372.362759:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766372.362767:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775766372.362812:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3480 x1862022114490112/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766372.362848:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766372.362852:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775766372.362854:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766372.362860:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3749
00000100:00080000:2.0:1775766372.362867:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766372.362870:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766372.362892:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766372.362906:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1c00 x1862022114490240/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775766372.362913:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114490112:192.168.202.110@tcp:250:kworker.0
00000100:00080000:2.0:1775766372.362942:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766372.362944:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775766372.362946:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766372.362950:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3749
00000100:00080000:2.0:1775766372.362956:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766372.362960:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766372.362967:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766372.362981:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2300 x1862022114490368/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766372.362989:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766372.362992:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775766372.362994:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766372.362997:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3749
00000100:00080000:2.0:1775766372.363002:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000800:00000100:0.0:1775766372.363009:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:2.0:1775766372.363009:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766372.363021:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775766372.363034:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3100 x1862022114490496/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766372.363042:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766372.363045:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1775766372.363047:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766372.363056:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3749
00000100:00080000:2.0:1775766372.363061:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1775766372.363065:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766372/real 1775766372]  req@ffff9f14852d3480 x1862022114490112/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766397 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766372.363066:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766372.363078:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775766372.363098:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1880 x1862022114490624/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775766372.363106:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3480 x1862022114490112/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766397 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766372.363135:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766372.363151:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775766372.363157:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766372.363174:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3749
00000100:00080000:2.0:1775766372.363178:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766372.363179:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766372.363183:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00080000:0.0:1775766372.363187:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000008:00100000:2.0:1775766372.363204:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775766372.363222:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114490112:192.168.202.110@tcp:250:kworker.0
00000100:00100000:2.0:1775766372.363228:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0e00 x1862022114490752/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766372.363243:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766372.363246:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775766372.363247:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775766372.363251:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114490240:192.168.202.110@tcp:38:kworker.0
00000100:00080000:2.0:1775766372.363252:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3749
00000100:00080000:2.0:1775766372.363263:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766372.363267:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766372.363272:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766372.363286:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3b80 x1862022114490880/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1775766372.363300:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:2.0:1775766372.363316:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766372.363320:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775766372.363322:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766372.363332:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3749
00000100:00100000:0.0:1775766372.363338:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766372/real 1775766372]  req@ffff9f14852d1c00 x1862022114490240/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766397 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766372.363338:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766372.363347:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766372.363358:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775766372.363379:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0000 x1862022114491008/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775766372.363422:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1c00 x1862022114490240/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766397 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766372.363435:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766372.363439:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766372.363443:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114490240:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766372.363455:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114490368:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766372.363487:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766372.363515:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766372/real 1775766372]  req@ffff9f14852d2300 x1862022114490368/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766397 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766372.363527:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2300 x1862022114490368/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766397 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766372.363533:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766372.363535:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766372.363538:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114490368:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766372.363542:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114490496:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766372.363585:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766372.363602:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766372/real 1775766372]  req@ffff9f14852d3100 x1862022114490496/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766397 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766372.363618:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3100 x1862022114490496/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766397 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766372.363635:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766372.363641:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766372.363646:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114490496:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775766372.363654:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114490624:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766372.363677:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766372.363688:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766372/real 1775766372]  req@ffff9f14852d1880 x1862022114490624/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766397 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766372.363701:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1880 x1862022114490624/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766397 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766372.363710:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766372.363713:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766372.363717:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114490624:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775766372.363723:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114490752:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766372.363743:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766372.363753:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766372/real 1775766372]  req@ffff9f14852d0e00 x1862022114490752/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766397 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766372.363766:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0e00 x1862022114490752/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766397 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766372.363775:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766372.363779:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766372.363782:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114490752:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766372.363823:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114490880:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766372.363844:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766372.363855:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766372/real 1775766372]  req@ffff9f14852d3b80 x1862022114490880/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766397 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766372.363867:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3b80 x1862022114490880/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766397 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766372.363875:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766372.363879:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766372.363882:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114490880:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766372.363889:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114491008:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766372.363905:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766372.363915:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766372/real 1775766372]  req@ffff9f14852d0000 x1862022114491008/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766397 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766372.363926:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0000 x1862022114491008/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766397 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766372.363935:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766372.363938:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766372.363942:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114491008:192.168.202.110@tcp:8:kworker.0
00010000:00010000:1.0:1775766372.618796:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775766373.386676:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766373.642785:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775766374.410684:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775766374.666782:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775766375.434703:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766375.690792:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775766376.458682:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766376.714851:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1775766377.482688:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766377.482697:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1775766377.482701:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775766377.482704:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:1.0:1775766377.482710:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3754
00000100:00080000:1.0:1775766377.482721:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766377.482728:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775766377.482764:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fea00 x1862022114491136/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766377.482788:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766377.482792:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775766377.482794:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766377.482799:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3754
00000100:00080000:1.0:1775766377.482803:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766377.482809:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766377.482821:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775766377.482825:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114491136:192.168.202.110@tcp:250:kworker.0
00000100:00100000:1.0:1775766377.482840:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff800 x1862022114491264/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766377.482856:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766377.482860:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775766377.482863:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766377.482868:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3754
00000100:00080000:1.0:1775766377.482881:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000800:00000100:0.0:1775766377.482888:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:1.0:1775766377.482888:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766377.482896:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775766377.482907:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdf80 x1862022114491392/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766377.482915:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766377.482918:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775766377.482920:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766377.482924:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3754
00000100:00100000:0.0:1775766377.482928:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766377/real 1775766377]  req@ffff9f14ab4fea00 x1862022114491136/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766402 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766377.482929:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766377.482936:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766377.482944:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775766377.482954:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fea00 x1862022114491136/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766402 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775766377.482956:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ffb80 x1862022114491520/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775766377.482977:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775766377.482988:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766377.482989:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:1.0:1775766377.482991:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1775766377.482996:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775766377.483000:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114491136:192.168.202.110@tcp:250:kworker.0
00000100:00080000:1.0:1775766377.483001:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3754
00000100:00080000:1.0:1775766377.483007:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766377.483012:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766377.483015:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775766377.483024:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fce00 x1862022114491648/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766377.483032:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1775766377.483037:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114491264:192.168.202.110@tcp:38:kworker.0
00000100:00080000:1.0:1775766377.483037:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775766377.483040:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766377.483048:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3754
00000100:00080000:1.0:1775766377.483055:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766377.483058:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766377.483061:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775766377.483071:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fca80 x1862022114491776/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766377.483078:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766377.483081:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775766377.483084:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000800:00000100:0.0:1775766377.483085:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:1.0:1775766377.483092:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3754
00000100:00080000:1.0:1775766377.483096:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766377.483098:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766377.483103:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775766377.483107:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766377/real 1775766377]  req@ffff9f14ab4ff800 x1862022114491264/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766402 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775766377.483115:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fed80 x1862022114491904/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775766377.483123:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff800 x1862022114491264/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766402 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766377.483124:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766377.483132:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775766377.483135:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766377.483140:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766377.483144:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00080000:1.0:1775766377.483150:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3754
00000100:00100000:0.0:1775766377.483151:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114491264:192.168.202.110@tcp:38:kworker.0
00000100:00080000:1.0:1775766377.483157:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1775766377.483162:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114491392:192.168.202.110@tcp:38:kworker.0
00000100:00080000:1.0:1775766377.483163:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766377.483170:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775766377.483183:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdc00 x1862022114492032/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1775766377.483191:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766377.483204:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766377/real 1775766377]  req@ffff9f14ab4fdf80 x1862022114491392/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766402 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766377.483214:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdf80 x1862022114491392/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766402 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766377.483220:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766377.483223:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766377.483225:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114491392:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766377.483229:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ffb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114491520:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766377.483241:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766377.483251:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766377/real 1775766377]  req@ffff9f14ab4ffb80 x1862022114491520/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766402 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766377.483291:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ffb80 x1862022114491520/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766402 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766377.483317:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766377.483323:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766377.483327:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ffb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114491520:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775766377.483346:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114491648:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766377.483386:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766377.483398:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766377/real 1775766377]  req@ffff9f14ab4fce00 x1862022114491648/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766402 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766377.483426:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fce00 x1862022114491648/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766402 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766377.483432:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766377.483435:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766377.483437:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114491648:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775766377.483441:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114491776:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766377.483455:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766377.483461:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766377/real 1775766377]  req@ffff9f14ab4fca80 x1862022114491776/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766402 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766377.483469:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fca80 x1862022114491776/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766402 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766377.483480:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766377.483483:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766377.483485:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114491776:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766377.483489:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114491904:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766377.483499:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766377.483504:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766377/real 1775766377]  req@ffff9f14ab4fed80 x1862022114491904/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766402 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766377.483512:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fed80 x1862022114491904/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766402 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766377.483518:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766377.483520:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766377.483522:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114491904:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766377.483526:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114492032:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766377.483539:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766377.483550:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766377/real 1775766377]  req@ffff9f14ab4fdc00 x1862022114492032/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766402 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766377.483563:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdc00 x1862022114492032/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766402 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766377.483619:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766377.483625:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766377.483631:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114492032:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775766377.738757:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766378.506613:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766378.762690:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766379.530667:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775766379.786791:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766380.554648:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766380.810699:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766381.578660:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775766381.834798:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1775766382.602718:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766382.602736:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00100000:0.0:1775766382.602742:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:1.0:1775766382.602742:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766382.602759:0:8:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3760
00000100:00080000:1.0:1775766382.602777:0:8:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766382.602788:0:8:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775766382.602851:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff100 x1862022114492160/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766382.602886:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766382.602892:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775766382.602895:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766382.602902:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3760
00000100:00080000:1.0:1775766382.602910:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766382.602916:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766382.602931:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775766382.602945:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114492160:192.168.202.110@tcp:250:kworker.0
00000100:00100000:1.0:1775766382.602952:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fd880 x1862022114492288/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766382.602967:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766382.602971:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775766382.602974:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766382.602979:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3760
00000100:00080000:1.0:1775766382.602984:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766382.602989:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766382.602995:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775766382.603007:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fe300 x1862022114492416/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766382.603019:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766382.603023:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775766382.603025:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766382.603031:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3760
00000800:00000100:0.0:1775766382.603034:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:1.0:1775766382.603035:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766382.603040:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766382.603046:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775766382.603058:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fc380 x1862022114492544/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766382.603078:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766382.603083:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1775766382.603085:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766382.603090:0:8:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3760
00000100:00100000:0.0:1775766382.603093:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766382/real 1775766382]  req@ffff9f14ab4ff100 x1862022114492160/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766407 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766382.603096:0:8:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766382.603101:0:8:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766382.603108:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775766382.603121:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff100 x1862022114492160/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766407 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775766382.603121:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdc00 x1862022114492672/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766382.603136:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766382.603139:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775766382.603141:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775766382.603144:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766382.603145:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:1.0:1775766382.603149:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3760
00000100:00100000:0.0:1775766382.603152:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114492160:192.168.202.110@tcp:250:kworker.0
00000100:00080000:1.0:1775766382.603154:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766382.603160:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766382.603166:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775766382.603179:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fed80 x1862022114492800/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775766382.603184:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114492288:192.168.202.110@tcp:38:kworker.0
00000100:00080000:1.0:1775766382.603194:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766382.603199:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775766382.603201:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766382.603207:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3760
00000100:00080000:1.0:1775766382.603213:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000800:00000100:0.0:1775766382.603216:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:1.0:1775766382.603218:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766382.603225:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775766382.603233:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766382/real 1775766382]  req@ffff9f14ab4fd880 x1862022114492288/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766407 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775766382.603239:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fca80 x1862022114492928/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775766382.603253:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fd880 x1862022114492288/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766407 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766382.603253:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766382.603259:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775766382.603262:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766382.603267:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766382.603272:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766382.603278:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114492288:192.168.202.110@tcp:38:kworker.0
00000100:00080000:1.0:1775766382.603282:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3760
00000100:00100000:0.0:1775766382.603286:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114492416:192.168.202.110@tcp:38:kworker.0
00000100:00080000:1.0:1775766382.603288:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766382.603293:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766382.603300:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775766382.603337:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fce00 x1862022114493056/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1775766382.603346:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766382.603363:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766382/real 1775766382]  req@ffff9f14ab4fe300 x1862022114492416/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766407 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766382.603382:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fe300 x1862022114492416/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766407 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766382.603394:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766382.603399:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766382.603404:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114492416:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766382.603413:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114492544:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766382.603438:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766382.603449:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766382/real 1775766382]  req@ffff9f14ab4fc380 x1862022114492544/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766407 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766382.603461:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fc380 x1862022114492544/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766407 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766382.603469:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766382.603472:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766382.603475:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114492544:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775766382.603480:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114492672:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766382.603494:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766382.603503:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766382/real 1775766382]  req@ffff9f14ab4fdc00 x1862022114492672/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766407 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766382.603514:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdc00 x1862022114492672/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766407 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766382.603521:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766382.603524:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766382.603526:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114492672:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775766382.603544:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114492800:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766382.603586:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766382.603597:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766382/real 1775766382]  req@ffff9f14ab4fed80 x1862022114492800/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766407 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766382.603609:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fed80 x1862022114492800/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766407 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766382.603617:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766382.603620:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766382.603623:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114492800:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766382.603628:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114492928:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766382.603643:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766382.603651:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766382/real 1775766382]  req@ffff9f14ab4fca80 x1862022114492928/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766407 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766382.603662:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fca80 x1862022114492928/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766407 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766382.603670:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766382.603673:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766382.603676:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114492928:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766382.603681:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114493056:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766382.603694:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766382.603701:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766382/real 1775766382]  req@ffff9f14ab4fce00 x1862022114493056/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766407 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766382.603711:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fce00 x1862022114493056/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766407 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766382.603719:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766382.603722:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766382.603724:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114493056:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775766382.858787:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766383.626656:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766383.882712:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766384.650713:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766384.906733:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766385.674643:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766385.930721:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766386.698741:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775766386.954739:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766387.722674:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:2.0:1775766387.722792:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766387.722800:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1775766387.722805:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766387.722815:0:8:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3765
00000100:00080000:2.0:1775766387.722827:0:8:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766387.722835:0:8:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775766387.722901:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2d80 x1862022114493184/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766387.722941:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766387.722957:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775766387.722959:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766387.722965:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3765
00000100:00080000:2.0:1775766387.722971:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766387.722977:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766387.723007:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766387.723022:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0380 x1862022114493312/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775766387.723028:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114493184:192.168.202.110@tcp:250:kworker.0
00000100:00080000:2.0:1775766387.723041:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766387.723045:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775766387.723046:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766387.723052:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3765
00000100:00080000:2.0:1775766387.723056:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766387.723060:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766387.723065:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766387.723087:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3800 x1862022114493440/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766387.723096:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766387.723099:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775766387.723101:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766387.723105:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3765
00000100:00080000:2.0:1775766387.723109:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766387.723111:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766387.723116:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:0.0:1775766387.723123:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766387.723131:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0700 x1862022114493568/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766387.723142:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766387.723145:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1775766387.723147:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766387.723151:0:8:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3765
00000100:00080000:2.0:1775766387.723154:0:8:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766387.723157:0:8:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766387.723163:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775766387.723176:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766387/real 1775766387]  req@ffff9f14852d2d80 x1862022114493184/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766412 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766387.723177:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2a00 x1862022114493696/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766387.723217:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766387.723228:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775766387.723230:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766387.723233:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3765
00000100:00080000:2.0:1775766387.723237:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766387.723241:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766387.723245:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775766387.723248:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2d80 x1862022114493184/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766412 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766387.723260:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1180 x1862022114493824/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775766387.723330:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766387.723333:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766387.723339:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:2.0:1775766387.723344:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00100000:0.0:1775766387.723349:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114493184:192.168.202.110@tcp:250:kworker.0
00000100:00080000:2.0:1775766387.723350:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766387.723364:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3765
00000100:00080000:2.0:1775766387.723369:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766387.723374:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766387.723380:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775766387.723385:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114493312:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766387.723397:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2680 x1862022114493952/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766387.723409:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766387.723413:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775766387.723416:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000800:00000100:0.0:1775766387.723421:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:2.0:1775766387.723427:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3765
00000100:00080000:2.0:1775766387.723433:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766387.723438:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766387.723443:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775766387.723445:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766387/real 1775766387]  req@ffff9f14852d0380 x1862022114493312/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766412 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766387.723458:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1f80 x1862022114494080/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775766387.723467:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0380 x1862022114493312/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766412 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766387.723485:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766387.723489:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766387.723493:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114493312:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766387.723501:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114493440:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766387.723520:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766387.723529:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766387/real 1775766387]  req@ffff9f14852d3800 x1862022114493440/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766412 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766387.723540:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3800 x1862022114493440/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766412 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766387.723548:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766387.723551:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766387.723580:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114493440:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766387.723607:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114493568:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766387.723634:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766387.723648:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766387/real 1775766387]  req@ffff9f14852d0700 x1862022114493568/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766412 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766387.723663:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0700 x1862022114493568/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766412 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766387.723674:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766387.723677:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766387.723681:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114493568:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775766387.723688:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114493696:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766387.723705:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766387.723743:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766387/real 1775766387]  req@ffff9f14852d2a00 x1862022114493696/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766412 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766387.723764:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2a00 x1862022114493696/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766412 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766387.723771:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766387.723774:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766387.723777:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114493696:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775766387.723782:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114493824:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766387.723798:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766387.723806:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766387/real 1775766387]  req@ffff9f14852d1180 x1862022114493824/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766412 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766387.723817:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1180 x1862022114493824/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766412 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766387.723824:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766387.723827:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766387.723829:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114493824:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766387.723849:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114493952:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766387.723871:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766387.723881:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766387/real 1775766387]  req@ffff9f14852d2680 x1862022114493952/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766412 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766387.723899:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2680 x1862022114493952/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766412 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766387.723911:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766387.723915:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766387.723920:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114493952:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766387.723929:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114494080:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766387.723957:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766387.723972:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766387/real 1775766387]  req@ffff9f14852d1f80 x1862022114494080/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766412 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766387.723990:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1f80 x1862022114494080/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766412 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766387.724012:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766387.724017:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766387.724022:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114494080:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775766387.978776:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766388.746766:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775766389.002743:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766389.770668:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766390.026727:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766390.794750:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766391.050856:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766391.818628:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766392.074712:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766392.842686:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:1.0:1775766392.842796:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766392.842806:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1775766392.842811:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766392.842823:0:8:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3770
00000100:00080000:1.0:1775766392.842838:0:8:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766392.842847:0:8:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775766392.842894:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ffb80 x1862022114494208/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766392.842933:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766392.842939:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775766392.842942:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766392.842947:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3770
00000100:00080000:1.0:1775766392.842951:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766392.842955:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766392.842979:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775766392.842998:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdf80 x1862022114494336/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775766392.843008:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ffb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114494208:192.168.202.110@tcp:250:kworker.0
00000100:00080000:1.0:1775766392.843024:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766392.843029:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775766392.843031:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766392.843038:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3770
00000100:00080000:1.0:1775766392.843043:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766392.843047:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766392.843052:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775766392.843068:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff800 x1862022114494464/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766392.843080:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766392.843085:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775766392.843087:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766392.843091:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3770
00000100:00080000:1.0:1775766392.843097:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000800:00000100:0.0:1775766392.843105:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:1.0:1775766392.843111:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766392.843120:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775766392.843144:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fea00 x1862022114494592/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766392.843158:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766392.843162:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1775766392.843165:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766392.843185:0:8:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3770
00000100:00080000:1.0:1775766392.843191:0:8:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766392.843195:0:8:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766392.843202:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775766392.843234:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766392/real 1775766392]  req@ffff9f14ab4ffb80 x1862022114494208/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766417 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775766392.843244:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fce00 x1862022114494720/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766392.843276:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766392.843280:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775766392.843282:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775766392.843289:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ffb80 x1862022114494208/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766417 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766392.843290:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3770
00000100:00080000:1.0:1775766392.843302:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766392.843314:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766392.843323:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00080000:0.0:1775766392.843324:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766392.843356:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775766392.843365:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fca80 x1862022114494848/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775766392.843370:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ffb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114494208:192.168.202.110@tcp:250:kworker.0
00000100:00080000:1.0:1775766392.843396:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766392.843401:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775766392.843404:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766392.843411:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3770
00000100:00080000:1.0:1775766392.843415:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766392.843420:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766392.843427:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775766392.843432:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114494336:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775766392.843461:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fed80 x1862022114494976/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766392.843475:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766392.843479:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775766392.843482:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766392.843486:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3770
00000100:00080000:1.0:1775766392.843501:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000800:00000100:0.0:1775766392.843508:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:1.0:1775766392.843515:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766392.843523:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775766392.843547:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdc00 x1862022114495104/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775766392.843552:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766392/real 1775766392]  req@ffff9f14ab4fdf80 x1862022114494336/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766417 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766392.843572:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdf80 x1862022114494336/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766417 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766392.843623:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766392.843630:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766392.843636:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114494336:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766392.843651:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114494464:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766392.843692:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766392.843718:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766392/real 1775766392]  req@ffff9f14ab4ff800 x1862022114494464/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766417 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766392.843742:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff800 x1862022114494464/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766417 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766392.843775:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766392.843780:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766392.843784:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114494464:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766392.843793:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114494592:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766392.843824:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766392.843838:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766392/real 1775766392]  req@ffff9f14ab4fea00 x1862022114494592/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766417 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766392.843853:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fea00 x1862022114494592/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766417 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766392.843865:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766392.843871:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766392.843879:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114494592:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775766392.843891:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114494720:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766392.843927:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766392.843949:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766392/real 1775766392]  req@ffff9f14ab4fce00 x1862022114494720/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766417 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766392.843974:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fce00 x1862022114494720/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766417 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766392.843994:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766392.843999:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766392.844003:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114494720:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775766392.844011:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114494848:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766392.844039:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766392.844053:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766392/real 1775766392]  req@ffff9f14ab4fca80 x1862022114494848/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766417 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766392.844067:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fca80 x1862022114494848/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766417 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766392.844077:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766392.844081:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766392.844085:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114494848:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766392.844110:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114494976:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766392.844129:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766392.844149:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766392/real 1775766392]  req@ffff9f14ab4fed80 x1862022114494976/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766417 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766392.844181:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fed80 x1862022114494976/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766417 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766392.844190:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766392.844193:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766392.844196:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114494976:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766392.844202:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114495104:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766392.844251:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766392.844265:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766392/real 1775766392]  req@ffff9f14ab4fdc00 x1862022114495104/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766417 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766392.844279:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdc00 x1862022114495104/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766417 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766392.844289:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766392.844292:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766392.844296:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114495104:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775766393.098801:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766393.866678:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775766394.122695:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766394.890661:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766395.146779:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766395.914686:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775766396.170773:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766396.938640:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766397.194694:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766397.962692:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:2.0:1775766397.962772:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766397.962781:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1775766397.962786:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766397.962797:0:8:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3775
00000100:00080000:2.0:1775766397.962810:0:8:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766397.962818:0:8:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775766397.962859:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1500 x1862022114495232/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766397.962895:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766397.962900:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775766397.962902:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766397.962907:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3775
00000100:00080000:2.0:1775766397.962910:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766397.962915:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766397.962927:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766397.962942:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0a80 x1862022114495360/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775766397.962949:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114495232:192.168.202.110@tcp:250:kworker.0
00000100:00080000:2.0:1775766397.962972:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766397.962978:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775766397.962981:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766397.962990:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3775
00000100:00080000:2.0:1775766397.962998:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766397.963004:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766397.963013:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766397.963037:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1f80 x1862022114495488/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766397.963055:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766397.963061:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775766397.963064:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766397.963072:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3775
00000100:00080000:2.0:1775766397.963081:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766397.963089:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766397.963098:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775766397.963099:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114495360:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766397.963120:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2680 x1862022114495616/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766397.963137:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766397.963141:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1775766397.963144:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766397.963149:0:8:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3775
00000100:00080000:2.0:1775766397.963155:0:8:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1775766397.963158:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114495488:192.168.202.110@tcp:38:kworker.0
00000100:00080000:2.0:1775766397.963192:0:8:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766397.963226:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775766397.963251:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1180 x1862022114495744/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766397.963272:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1775766397.963274:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114495616:192.168.202.110@tcp:8:kworker.0
00000100:00080000:2.0:1775766397.963286:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775766397.963292:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766397.963301:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3775
00000100:00080000:2.0:1775766397.963311:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766397.963321:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766397.963332:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775766397.963334:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114495744:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775766397.963365:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2a00 x1862022114495872/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766397.963385:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766397.963391:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775766397.963394:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775766397.963401:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114495872:192.168.202.110@tcp:38:kworker.0
00000100:00080000:2.0:1775766397.963408:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3775
00000100:00080000:2.0:1775766397.963423:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766397.963430:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766397.963440:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766397.963466:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0700 x1862022114496000/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766397.963498:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766397.963506:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775766397.963509:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766397.963518:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3775
00000100:00080000:2.0:1775766397.963526:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1775766397.963528:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114496000:192.168.202.110@tcp:38:kworker.0
00000100:00080000:2.0:1775766397.963540:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766397.963618:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775766397.963650:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3800 x1862022114496128/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775766397.964038:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114496128:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775766398.218764:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766398.986689:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766399.242784:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766400.010700:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766400.266831:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766401.034680:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000400:00000100:2.0:1775766401.035089:0:2391:0:(lib-socket.c:454:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.202.110:988
00000400:00000100:2.0:1775766401.035199:0:2391:0:(acceptor.c:111:lnet_connect_console_error()) Connection to 192.168.202.110@tcp at host 192.168.202.110 was unreachable: the network or that node may be down, or Lustre may be misconfigured.
00000800:00000100:2.0:1775766401.035229:0:2391:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:2.0:1775766401.035310:0:2391:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:2.0:1775766401.035340:0:2391:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:2.0:1775766401.035367:0:2391:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:2.0:1775766401.035384:0:2391:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:2.0:1775766401.035401:0:2391:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:2.0:1775766401.035415:0:2391:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:2.0:1775766401.035434:0:2391:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000100:00100000:1.0:1775766401.035723:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766397/real 1775766401]  req@ffff9f14852d3800 x1862022114496128/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766422 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775766401.035787:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3800 x1862022114496128/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766422 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766401.035813:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775766401.035827:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775766401.035837:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114496128:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775766401.035850:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766397/real 1775766401]  req@ffff9f14852d0700 x1862022114496000/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766422 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775766401.035874:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0700 x1862022114496000/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766422 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766401.035886:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775766401.035891:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775766401.035896:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114496000:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775766401.035908:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766397/real 1775766401]  req@ffff9f14852d2a00 x1862022114495872/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766422 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775766401.035927:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2a00 x1862022114495872/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766422 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766401.035939:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775766401.035944:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775766401.035950:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114495872:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775766401.035962:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766397/real 1775766401]  req@ffff9f14852d1180 x1862022114495744/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766422 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775766401.035983:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1180 x1862022114495744/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766422 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766401.035995:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775766401.036000:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775766401.036006:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114495744:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775766401.036016:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766397/real 1775766401]  req@ffff9f14852d2680 x1862022114495616/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766422 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775766401.036034:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2680 x1862022114495616/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766422 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766401.036045:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775766401.036049:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775766401.036054:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114495616:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775766401.036064:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766397/real 1775766401]  req@ffff9f14852d0a80 x1862022114495360/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766422 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775766401.036081:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0a80 x1862022114495360/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766422 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766401.036092:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775766401.036097:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775766401.036101:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114495360:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775766401.036111:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766397/real 1775766401]  req@ffff9f14852d1f80 x1862022114495488/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766422 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775766401.036156:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1f80 x1862022114495488/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766422 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766401.036169:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775766401.036175:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775766401.036181:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114495488:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775766401.036194:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766397/real 1775766401]  req@ffff9f14852d1500 x1862022114495232/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766422 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775766401.036218:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1500 x1862022114495232/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766422 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766401.036232:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775766401.036242:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775766401.036247:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114495232:192.168.202.110@tcp:250:kworker.0
00010000:00010000:1.0:1775766401.290786:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766402.058660:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766402.314694:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766403.082701:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775766403.082741:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766403.082754:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775766403.082761:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766403.082776:0:8:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3780
00000100:00080000:3.0:1775766403.082793:0:8:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766403.082803:0:8:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775766403.082860:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744a80 x1862022114496256/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766403.082900:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766403.082905:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775766403.082908:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766403.082913:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3780
00000100:00080000:3.0:1775766403.082917:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766403.082921:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766403.082936:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766403.082973:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745880 x1862022114496384/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:1.0:1775766403.082987:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114496256:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775766403.082993:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766403.083001:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775766403.083004:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766403.083010:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3780
00000100:00080000:3.0:1775766403.083015:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766403.083020:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766403.083028:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766403.083042:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747480 x1862022114496512/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766403.083055:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766403.083059:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775766403.083062:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766403.083067:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3780
00000100:00080000:3.0:1775766403.083072:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766403.083076:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766403.083081:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766403.083093:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746300 x1862022114496640/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766403.083105:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766403.083149:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775766403.083154:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000800:00000100:1.0:1775766403.083158:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775766403.083167:0:8:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3780
00000100:00080000:3.0:1775766403.083176:0:8:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766403.083184:0:8:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766403.083195:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766403.083218:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746a00 x1862022114496768/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766403.083237:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:1.0:1775766403.083243:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766403/real 1775766403]  req@ffff9f1485744a80 x1862022114496256/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766428 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766403.083245:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775766403.083250:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766403.083267:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3780
00000100:00080000:3.0:1775766403.083277:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766403.083288:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766403.083299:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775766403.083309:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744a80 x1862022114496256/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766428 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766403.083333:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00100000:3.0:1775766403.083343:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744380 x1862022114496896/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766403.083346:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775766403.083361:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114496256:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775766403.083385:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766403.083407:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775766403.083410:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:1.0:1775766403.083414:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114496384:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775766403.083424:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3780
00000100:00080000:3.0:1775766403.083434:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766403.083442:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766403.083452:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000800:00000100:1.0:1775766403.083472:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766403.083479:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744e00 x1862022114497024/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766403.083501:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:1.0:1775766403.083503:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766403/real 1775766403]  req@ffff9f1485745880 x1862022114496384/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766428 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766403.083516:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775766403.083525:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766403.083550:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3780
00000100:00080000:3.0:1775766403.083629:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:1.0:1775766403.083634:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745880 x1862022114496384/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766428 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766403.083642:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766403.083660:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00080000:1.0:1775766403.083664:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775766403.083674:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775766403.083682:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114496384:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766403.083691:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747b80 x1862022114497152/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:1.0:1775766403.083702:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114496512:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775766403.083755:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775766403.083776:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766403/real 1775766403]  req@ffff9f1485747480 x1862022114496512/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766428 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775766403.083799:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747480 x1862022114496512/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766428 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766403.083812:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775766403.083819:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775766403.083825:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114496512:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775766403.083837:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114496640:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775766403.083876:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775766403.083901:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766403/real 1775766403]  req@ffff9f1485746300 x1862022114496640/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766428 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775766403.083923:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746300 x1862022114496640/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766428 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766403.083949:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775766403.083956:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775766403.083962:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114496640:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775766403.083972:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114496768:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775766403.084007:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775766403.084026:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766403/real 1775766403]  req@ffff9f1485746a00 x1862022114496768/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766428 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775766403.084046:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746a00 x1862022114496768/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766428 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766403.084060:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775766403.084066:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775766403.084071:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114496768:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775766403.084082:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114496896:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775766403.084138:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775766403.084158:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766403/real 1775766403]  req@ffff9f1485744380 x1862022114496896/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766428 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775766403.084179:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744380 x1862022114496896/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766428 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766403.084193:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775766403.084198:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775766403.084206:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114496896:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775766403.084243:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114497024:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775766403.084279:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775766403.084297:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766403/real 1775766403]  req@ffff9f1485744e00 x1862022114497024/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766428 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775766403.084318:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744e00 x1862022114497024/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766428 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766403.084332:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775766403.084337:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775766403.084342:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114497024:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775766403.084353:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114497152:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775766403.084387:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775766403.084406:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766403/real 1775766403]  req@ffff9f1485747b80 x1862022114497152/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766428 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775766403.084425:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747b80 x1862022114497152/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766428 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766403.084439:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775766403.084445:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775766403.084450:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114497152:192.168.202.110@tcp:8:kworker.0
00010000:00010000:3.0:1775766403.338699:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766404.106700:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766404.362823:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766405.130654:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766405.386787:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766406.154680:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766406.410737:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766407.178660:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766407.434753:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766408.202653:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:1.0:1775766408.202659:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766408.202664:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1775766408.202669:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766408.202687:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3785
00000100:00080000:1.0:1775766408.202698:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766408.202706:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775766408.202747:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fc380 x1862022114497280/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766408.202772:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766408.202775:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775766408.202776:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766408.202781:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3785
00000100:00080000:1.0:1775766408.202785:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766408.202789:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766408.202798:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775766408.202809:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fe300 x1862022114497408/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775766408.202818:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114497280:192.168.202.110@tcp:250:kworker.0
00000100:00080000:1.0:1775766408.202824:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766408.202828:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775766408.202830:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766408.202834:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3785
00000100:00080000:1.0:1775766408.202837:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766408.202840:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766408.202845:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775766408.202859:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fd880 x1862022114497536/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766408.202887:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766408.202890:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775766408.202891:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766408.202894:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3785
00000100:00080000:1.0:1775766408.202897:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766408.202902:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000800:00000100:3.0:1775766408.202905:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000008:00100000:1.0:1775766408.202907:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775766408.202916:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff100 x1862022114497664/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766408.202922:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766408.202925:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1775766408.202926:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766408.202929:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3785
00000100:00080000:1.0:1775766408.202934:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766408.202937:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766408.202942:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775766408.202971:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdc00 x1862022114497792/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766408.202981:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766408.202985:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775766408.202987:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766408.202992:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3785
00000100:00080000:1.0:1775766408.202999:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766408.203006:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766408.203013:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775766408.203029:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fed80 x1862022114497920/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766408.203036:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766408.203038:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775766408.203039:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766408.203042:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3785
00000100:00080000:1.0:1775766408.203044:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766408.203046:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766408.203049:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775766408.203074:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fca80 x1862022114498048/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766408.203081:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766408.203084:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775766408.203085:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766408.203087:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3785
00000100:00080000:1.0:1775766408.203090:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766408.203092:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766408.203118:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775766408.203126:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fce00 x1862022114498176/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775766408.203159:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766408/real 1775766408]  req@ffff9f14ab4fc380 x1862022114497280/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766433 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766408.203182:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fc380 x1862022114497280/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766433 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766408.203223:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766408.203228:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766408.203232:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114497280:192.168.202.110@tcp:250:kworker.0
00000100:00100000:3.0:1775766408.203250:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114497408:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775766408.203276:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766408.203286:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766408/real 1775766408]  req@ffff9f14ab4fe300 x1862022114497408/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766433 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766408.203294:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fe300 x1862022114497408/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766433 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766408.203304:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766408.203307:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766408.203309:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114497408:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766408.203313:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114497536:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775766408.203323:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766408.203329:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766408/real 1775766408]  req@ffff9f14ab4fd880 x1862022114497536/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766433 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766408.203336:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fd880 x1862022114497536/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766433 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766408.203341:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766408.203343:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766408.203345:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114497536:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766408.203349:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114497664:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775766408.203358:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766408.203364:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766408/real 1775766408]  req@ffff9f14ab4ff100 x1862022114497664/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766433 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766408.203370:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff100 x1862022114497664/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766433 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766408.203376:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766408.203378:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766408.203380:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114497664:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775766408.203384:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114497792:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775766408.203393:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766408.203398:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766408/real 1775766408]  req@ffff9f14ab4fdc00 x1862022114497792/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766433 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766408.203405:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdc00 x1862022114497792/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766433 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766408.203410:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766408.203411:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766408.203413:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114497792:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775766408.203416:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114497920:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775766408.203425:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766408.203430:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766408/real 1775766408]  req@ffff9f14ab4fed80 x1862022114497920/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766433 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766408.203437:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fed80 x1862022114497920/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766433 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766408.203441:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766408.203443:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766408.203445:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114497920:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766408.203448:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114498048:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775766408.203458:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766408.203464:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766408/real 1775766408]  req@ffff9f14ab4fca80 x1862022114498048/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766433 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766408.203471:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fca80 x1862022114498048/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766433 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766408.203475:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766408.203477:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766408.203479:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114498048:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766408.203482:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114498176:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775766408.203491:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766408.203496:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766408/real 1775766408]  req@ffff9f14ab4fce00 x1862022114498176/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766433 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766408.203508:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fce00 x1862022114498176/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766433 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766408.203517:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766408.203520:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766408.203523:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114498176:192.168.202.110@tcp:8:kworker.0
00010000:00010000:1.0:1775766408.458738:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766409.226627:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775766409.482720:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766410.250684:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775766410.506771:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766411.274652:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766411.530760:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766412.298668:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766412.554733:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1775766413.322614:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766413.322623:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1775766413.322628:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775766413.322631:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:1.0:1775766413.322645:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3790
00000100:00080000:1.0:1775766413.322679:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766413.322690:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775766413.322739:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fea00 x1862022114498304/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766413.322775:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766413.322779:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775766413.322782:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766413.322789:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3790
00000100:00080000:1.0:1775766413.322795:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766413.322802:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766413.322816:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775766413.322850:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff800 x1862022114498432/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766413.322862:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766413.322866:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775766413.322868:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766413.322880:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3790
00000100:00080000:1.0:1775766413.322884:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766413.322888:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766413.322892:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775766413.322904:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdf80 x1862022114498560/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775766413.322905:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114498304:192.168.202.110@tcp:250:kworker.0
00000100:00080000:1.0:1775766413.322917:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766413.322920:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775766413.322921:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766413.322924:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3790
00000100:00080000:1.0:1775766413.322926:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766413.322928:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766413.322932:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775766413.322939:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ffb80 x1862022114498688/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766413.322945:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766413.322947:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1775766413.322949:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766413.322951:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3790
00000100:00080000:1.0:1775766413.322954:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766413.322957:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766413.322959:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775766413.322966:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fce00 x1862022114498816/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766413.322972:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766413.322975:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775766413.322976:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766413.322981:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3790
00000800:00000100:3.0:1775766413.322982:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:1.0:1775766413.322985:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766413.322989:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766413.322992:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775766413.323012:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fca80 x1862022114498944/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766413.323020:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766413.323024:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775766413.323026:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766413.323031:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3790
00000100:00080000:1.0:1775766413.323036:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766413.323041:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775766413.323041:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766413/real 1775766413]  req@ffff9f14ab4fea00 x1862022114498304/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766438 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:1.0:1775766413.323051:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775766413.323070:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fed80 x1862022114499072/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775766413.323075:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fea00 x1862022114498304/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766438 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766413.323092:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766413.323096:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775766413.323098:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766413.323104:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3790
00000100:00080000:3.0:1775766413.323104:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775766413.323109:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766413.323112:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:1.0:1775766413.323116:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775766413.323120:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114498304:192.168.202.110@tcp:250:kworker.0
00000008:00100000:1.0:1775766413.323122:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766413.323134:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114498432:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775766413.323137:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdc00 x1862022114499200/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:3.0:1775766413.323171:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766413.323185:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766413/real 1775766413]  req@ffff9f14ab4ff800 x1862022114498432/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766438 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766413.323197:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff800 x1862022114498432/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766438 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766413.323204:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766413.323206:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766413.323208:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114498432:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766413.323228:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114498560:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775766413.323250:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766413.323262:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766413/real 1775766413]  req@ffff9f14ab4fdf80 x1862022114498560/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766438 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766413.323274:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdf80 x1862022114498560/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766438 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766413.323290:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766413.323293:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766413.323297:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114498560:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766413.323301:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ffb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114498688:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775766413.323317:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766413.323324:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766413/real 1775766413]  req@ffff9f14ab4ffb80 x1862022114498688/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766438 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766413.323332:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ffb80 x1862022114498688/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766438 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766413.323338:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766413.323341:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766413.323343:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ffb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114498688:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775766413.323346:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114498816:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775766413.323356:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766413.323361:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766413/real 1775766413]  req@ffff9f14ab4fce00 x1862022114498816/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766438 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766413.323369:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fce00 x1862022114498816/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766438 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766413.323373:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766413.323375:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766413.323378:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114498816:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775766413.323381:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114498944:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775766413.323391:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766413.323396:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766413/real 1775766413]  req@ffff9f14ab4fca80 x1862022114498944/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766438 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766413.323403:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fca80 x1862022114498944/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766438 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766413.323408:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766413.323409:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766413.323411:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114498944:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766413.323414:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114499072:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775766413.323423:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766413.323428:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766413/real 1775766413]  req@ffff9f14ab4fed80 x1862022114499072/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766438 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766413.323435:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fed80 x1862022114499072/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766438 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766413.323440:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766413.323441:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766413.323443:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114499072:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766413.323446:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114499200:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775766413.323455:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766413.323461:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766413/real 1775766413]  req@ffff9f14ab4fdc00 x1862022114499200/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766438 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766413.323468:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdc00 x1862022114499200/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766438 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766413.323473:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766413.323475:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766413.323477:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114499200:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775766413.578856:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766414.346731:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766414.602817:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766415.370696:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775766415.626794:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766416.394652:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775766416.650732:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766417.418617:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775766417.674704:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766418.442645:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:1.0:1775766418.442724:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766418.442735:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1775766418.442741:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766418.442754:0:8:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3795
00000100:00080000:1.0:1775766418.442769:0:8:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766418.442778:0:8:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775766418.442833:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff100 x1862022114499328/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766418.442870:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766418.442885:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775766418.442887:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766418.442891:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3795
00000100:00080000:1.0:1775766418.442894:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766418.442897:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766418.442917:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775766418.442934:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fd880 x1862022114499456/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766418.442945:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:3.0:1775766418.442964:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114499328:192.168.202.110@tcp:250:kworker.0
00000100:00080000:1.0:1775766418.442970:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775766418.442978:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766418.443001:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3795
00000100:00080000:1.0:1775766418.443004:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766418.443010:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766418.443016:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775766418.443034:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fe300 x1862022114499584/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766418.443053:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766418.443057:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775766418.443058:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766418.443061:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3795
00000100:00080000:1.0:1775766418.443063:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766418.443066:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766418.443069:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775766418.443079:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fc380 x1862022114499712/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766418.443094:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000800:00000100:3.0:1775766418.443095:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:1.0:1775766418.443105:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1775766418.443107:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766418.443113:0:8:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3795
00000100:00080000:1.0:1775766418.443117:0:8:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766418.443122:0:8:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766418.443128:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775766418.443145:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdc00 x1862022114499840/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766418.443165:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766418.443168:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775766418.443170:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766418.443174:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3795
00000100:00080000:1.0:1775766418.443179:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766418.443183:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766418.443188:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766418.443193:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766418/real 1775766418]  req@ffff9f14ab4ff100 x1862022114499328/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766443 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775766418.443203:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fed80 x1862022114499968/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766418.443223:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766418.443227:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775766418.443229:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766418.443232:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3795
00000100:00080000:1.0:1775766418.443245:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:3.0:1775766418.443261:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff100 x1862022114499328/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766443 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766418.443275:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766418.443291:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00080000:3.0:1775766418.443303:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00100000:1.0:1775766418.443309:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fca80 x1862022114500096/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766418.443311:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766418.443330:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114499328:192.168.202.110@tcp:250:kworker.0
00000100:00080000:1.0:1775766418.443341:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766418.443345:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775766418.443348:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766418.443354:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3795
00000100:00080000:1.0:1775766418.443358:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766418.443362:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766418.443367:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766418.443384:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114499456:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775766418.443390:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fce00 x1862022114500224/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:3.0:1775766418.443419:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766418.443432:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766418/real 1775766418]  req@ffff9f14ab4fd880 x1862022114499456/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766443 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766418.443462:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fd880 x1862022114499456/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766443 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766418.443471:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766418.443474:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766418.443477:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114499456:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766418.443483:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114499584:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775766418.443497:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766418.443504:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766418/real 1775766418]  req@ffff9f14ab4fe300 x1862022114499584/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766443 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766418.443514:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fe300 x1862022114499584/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766443 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766418.443520:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766418.443523:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766418.443525:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114499584:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766418.443530:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114499712:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775766418.443543:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766418.443550:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766418/real 1775766418]  req@ffff9f14ab4fc380 x1862022114499712/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766443 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766418.443559:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fc380 x1862022114499712/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766443 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766418.443572:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766418.443574:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766418.443577:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114499712:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775766418.443581:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114499840:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775766418.443627:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766418.443636:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766418/real 1775766418]  req@ffff9f14ab4fdc00 x1862022114499840/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766443 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766418.443645:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdc00 x1862022114499840/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766443 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766418.443651:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766418.443654:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766418.443656:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114499840:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775766418.443661:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114499968:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775766418.443682:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766418.443695:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766418/real 1775766418]  req@ffff9f14ab4fed80 x1862022114499968/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766443 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766418.443733:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fed80 x1862022114499968/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766443 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766418.443761:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766418.443768:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766418.443776:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114499968:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766418.443788:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114500096:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775766418.443824:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766418.443847:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766418/real 1775766418]  req@ffff9f14ab4fca80 x1862022114500096/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766443 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766418.443884:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fca80 x1862022114500096/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766443 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766418.443890:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766418.443893:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766418.443896:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114500096:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766418.443925:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114500224:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775766418.443940:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766418.443968:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766418/real 1775766418]  req@ffff9f14ab4fce00 x1862022114500224/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766443 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766418.443991:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fce00 x1862022114500224/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766443 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766418.443998:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766418.444000:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766418.444003:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114500224:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775766418.698695:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766419.466652:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766419.722735:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766420.490631:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766420.746765:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766421.514653:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766421.770777:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775766422.538668:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775766422.794696:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775766423.562690:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:2.0:1775766423.562779:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766423.562789:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1775766423.562796:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766423.562808:0:8:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3801
00000100:00080000:2.0:1775766423.562824:0:8:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766423.562832:0:8:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775766423.562884:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0380 x1862022114500352/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766423.562941:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766423.562945:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775766423.562947:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766423.562952:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3801
00000100:00080000:2.0:1775766423.562956:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766423.562960:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766423.562973:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766423.562988:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2d80 x1862022114500480/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766423.562999:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766423.563002:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775766423.563004:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766423.563009:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3801
00000100:00080000:2.0:1775766423.563012:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766423.563019:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775766423.563020:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114500352:192.168.202.110@tcp:250:kworker.0
00000008:00100000:2.0:1775766423.563024:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766423.563037:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0000 x1862022114500608/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766423.563046:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766423.563049:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775766423.563050:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766423.563054:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3801
00000100:00080000:2.0:1775766423.563057:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766423.563060:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766423.563064:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775766423.563076:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3b80 x1862022114500736/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766423.563085:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766423.563088:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1775766423.563090:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766423.563102:0:8:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3801
00000100:00080000:2.0:1775766423.563106:0:8:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766423.563110:0:8:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766423.563116:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775766423.563127:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0e00 x1862022114500864/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:3.0:1775766423.563132:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:2.0:1775766423.563143:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766423.563147:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775766423.563149:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766423.563154:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3801
00000100:00080000:2.0:1775766423.563157:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766423.563162:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766423.563167:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766423.563176:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1880 x1862022114500992/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766423.563183:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766423.563186:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775766423.563188:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766423.563196:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3801
00000100:00100000:3.0:1775766423.563197:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766423/real 1775766423]  req@ffff9f14852d0380 x1862022114500352/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766448 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766423.563200:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766423.563209:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766423.563218:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766423.563231:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3100 x1862022114501120/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775766423.563236:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0380 x1862022114500352/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766448 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766423.563260:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766423.563264:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775766423.563266:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766423.563267:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766423.563272:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3801
00000100:00080000:2.0:1775766423.563276:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766423.563278:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:2.0:1775766423.563283:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775766423.563288:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114500352:192.168.202.110@tcp:250:kworker.0
00000008:00100000:2.0:1775766423.563290:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766423.563301:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114500480:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766423.563307:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2300 x1862022114501248/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:3.0:1775766423.563335:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766423.563349:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766423/real 1775766423]  req@ffff9f14852d2d80 x1862022114500480/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766448 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766423.563361:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2d80 x1862022114500480/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766448 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766423.563369:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766423.563372:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766423.563374:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114500480:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766423.563401:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114500608:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775766423.563415:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766423.563422:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766423/real 1775766423]  req@ffff9f14852d0000 x1862022114500608/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766448 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766423.563432:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0000 x1862022114500608/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766448 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766423.563445:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766423.563448:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766423.563451:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114500608:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766423.563456:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114500736:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775766423.563469:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766423.563476:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766423/real 1775766423]  req@ffff9f14852d3b80 x1862022114500736/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766448 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766423.563485:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3b80 x1862022114500736/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766448 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766423.563492:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766423.563494:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766423.563497:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114500736:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775766423.563501:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114500864:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775766423.563513:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766423.563520:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766423/real 1775766423]  req@ffff9f14852d0e00 x1862022114500864/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766448 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766423.563530:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0e00 x1862022114500864/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766448 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766423.563536:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766423.563538:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766423.563541:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114500864:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775766423.563546:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114500992:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775766423.563598:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766423.563608:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766423/real 1775766423]  req@ffff9f14852d1880 x1862022114500992/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766448 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766423.563618:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1880 x1862022114500992/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766448 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766423.563625:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766423.563628:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766423.563630:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114500992:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766423.563635:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114501120:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775766423.563649:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766423.563657:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766423/real 1775766423]  req@ffff9f14852d3100 x1862022114501120/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766448 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766423.563666:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3100 x1862022114501120/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766448 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766423.563672:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766423.563674:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766423.563677:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114501120:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766423.563681:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114501248:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775766423.563693:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766423.563700:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766423/real 1775766423]  req@ffff9f14852d2300 x1862022114501248/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766448 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766423.563710:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2300 x1862022114501248/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766448 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766423.563716:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766423.563718:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766423.563721:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114501248:192.168.202.110@tcp:8:kworker.0
00010000:00010000:3.0:1775766423.818749:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775766424.586635:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775766424.842695:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775766425.610634:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766425.866735:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775766426.634627:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766426.890665:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775766427.658650:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766427.914698:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1775766428.682640:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766428.682648:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1775766428.682652:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00100000:2.0:1775766428.682653:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:1.0:1775766428.682661:0:8:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3806
00000100:00080000:1.0:1775766428.682671:0:8:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766428.682677:0:8:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775766428.682711:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ffb80 x1862022114501376/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766428.682739:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766428.682741:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775766428.682743:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766428.682746:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3806
00000100:00080000:1.0:1775766428.682748:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766428.682750:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766428.682758:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775766428.682767:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdf80 x1862022114501504/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766428.682773:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766428.682776:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775766428.682777:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766428.682779:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3806
00000100:00080000:1.0:1775766428.682781:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766428.682783:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766428.682785:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775766428.682791:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff800 x1862022114501632/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766428.682797:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766428.682799:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775766428.682800:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766428.682802:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3806
00000100:00080000:1.0:1775766428.682805:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766428.682807:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766428.682810:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775766428.682816:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fea00 x1862022114501760/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775766428.682821:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ffb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114501376:192.168.202.110@tcp:250:kworker.0
00000100:00080000:1.0:1775766428.682828:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766428.682830:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1775766428.682832:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766428.682835:0:8:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3806
00000100:00080000:1.0:1775766428.682837:0:8:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766428.682839:0:8:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766428.682855:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775766428.682864:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fce00 x1862022114501888/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766428.682870:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766428.682873:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775766428.682874:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766428.682877:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3806
00000100:00080000:1.0:1775766428.682880:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766428.682883:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766428.682886:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775766428.682894:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fca80 x1862022114502016/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:3.0:1775766428.682901:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:1.0:1775766428.682907:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766428.682909:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775766428.682911:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766428.682915:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3806
00000100:00080000:1.0:1775766428.682918:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766428.682921:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766428.682923:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775766428.682931:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fed80 x1862022114502144/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766428.682936:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766428.682939:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775766428.682940:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:3.0:1775766428.682943:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766428/real 1775766428]  req@ffff9f14ab4ffb80 x1862022114501376/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766453 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766428.682946:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3806
00000100:00080000:1.0:1775766428.682951:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766428.682956:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766428.682962:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766428.682970:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ffb80 x1862022114501376/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766453 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775766428.682971:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdc00 x1862022114502272/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766428.683007:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766428.683012:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766428.683016:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ffb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114501376:192.168.202.110@tcp:250:kworker.0
00000100:00100000:3.0:1775766428.683022:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114501504:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775766428.683038:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766428.683045:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766428/real 1775766428]  req@ffff9f14ab4fdf80 x1862022114501504/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766453 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766428.683058:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdf80 x1862022114501504/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766453 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766428.683072:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766428.683074:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766428.683077:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114501504:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766428.683081:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114501632:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775766428.683092:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766428.683098:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766428/real 1775766428]  req@ffff9f14ab4ff800 x1862022114501632/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766453 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766428.683105:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff800 x1862022114501632/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766453 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766428.683110:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766428.683112:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766428.683114:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114501632:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766428.683133:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114501760:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775766428.683143:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766428.683148:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766428/real 1775766428]  req@ffff9f14ab4fea00 x1862022114501760/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766453 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766428.683156:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fea00 x1862022114501760/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766453 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766428.683165:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766428.683167:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766428.683169:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114501760:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775766428.683173:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114501888:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775766428.683181:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766428.683186:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766428/real 1775766428]  req@ffff9f14ab4fce00 x1862022114501888/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766453 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766428.683193:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fce00 x1862022114501888/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766453 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766428.683198:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766428.683200:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766428.683201:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114501888:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775766428.683204:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114502016:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775766428.683213:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766428.683218:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766428/real 1775766428]  req@ffff9f14ab4fca80 x1862022114502016/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766453 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766428.683225:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fca80 x1862022114502016/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766453 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766428.683230:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766428.683231:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766428.683234:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114502016:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766428.683237:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114502144:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775766428.683246:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766428.683251:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766428/real 1775766428]  req@ffff9f14ab4fed80 x1862022114502144/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766453 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766428.683258:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fed80 x1862022114502144/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766453 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766428.683263:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766428.683265:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766428.683266:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114502144:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766428.683269:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114502272:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775766428.683278:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766428.683283:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766428/real 1775766428]  req@ffff9f14ab4fdc00 x1862022114502272/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766453 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766428.683289:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdc00 x1862022114502272/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766453 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766428.683294:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766428.683296:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766428.683298:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114502272:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775766428.938749:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766429.706707:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766429.962734:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766430.730650:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775766430.986689:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766431.754676:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775766432.010790:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766432.778697:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766433.034777:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1775766433.802657:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766433.802667:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1775766433.802672:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775766433.802674:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:1.0:1775766433.802681:0:8:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3811
00000100:00080000:1.0:1775766433.802693:0:8:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766433.802700:0:8:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775766433.802755:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fc380 x1862022114502400/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766433.802814:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766433.802817:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775766433.802818:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766433.802822:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3811
00000100:00080000:1.0:1775766433.802825:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766433.802829:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766433.802837:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775766433.802849:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fe300 x1862022114502528/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775766433.802854:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114502400:192.168.202.110@tcp:250:kworker.0
00000100:00080000:1.0:1775766433.802860:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766433.802863:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775766433.802865:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766433.802868:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3811
00000100:00080000:1.0:1775766433.802870:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766433.802872:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766433.802875:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775766433.802883:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fd880 x1862022114502656/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766433.802890:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766433.802893:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775766433.802894:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766433.802896:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3811
00000100:00080000:1.0:1775766433.802898:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766433.802901:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766433.802904:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775766433.802911:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff100 x1862022114502784/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:3.0:1775766433.802917:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:1.0:1775766433.802924:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766433.802927:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1775766433.802929:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766433.802932:0:8:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3811
00000100:00080000:1.0:1775766433.802937:0:8:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766433.802939:0:8:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766433.802942:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775766433.802949:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdc00 x1862022114502912/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766433.802955:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766433.802958:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775766433.802959:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766433.802964:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3811
00000100:00100000:3.0:1775766433.802965:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766433/real 1775766433]  req@ffff9f14ab4fc380 x1862022114502400/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766458 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766433.802966:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766433.802972:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766433.802977:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775766433.802988:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fed80 x1862022114503040/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775766433.802993:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fc380 x1862022114502400/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766458 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766433.803006:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766433.803011:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775766433.803011:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775766433.803012:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766433.803019:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:1.0:1775766433.803020:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3811
00000100:00080000:1.0:1775766433.803025:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:3.0:1775766433.803027:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114502400:192.168.202.110@tcp:250:kworker.0
00000100:00080000:1.0:1775766433.803031:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766433.803053:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775766433.803061:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fca80 x1862022114503168/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766433.803069:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:3.0:1775766433.803069:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114502528:192.168.202.110@tcp:38:kworker.0
00000100:00080000:1.0:1775766433.803077:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775766433.803080:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766433.803084:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3811
00000100:00080000:1.0:1775766433.803087:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766433.803091:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766433.803095:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:3.0:1775766433.803099:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775766433.803106:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fce00 x1862022114503296/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775766433.803114:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766433/real 1775766433]  req@ffff9f14ab4fe300 x1862022114502528/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766458 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766433.803126:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fe300 x1862022114502528/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766458 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766433.803134:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766433.803137:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766433.803140:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114502528:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766433.803161:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114502656:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775766433.803173:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766433.803179:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766433/real 1775766433]  req@ffff9f14ab4fd880 x1862022114502656/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766458 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766433.803191:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fd880 x1862022114502656/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766458 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766433.803196:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766433.803198:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766433.803200:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114502656:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766433.803203:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114502784:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775766433.803214:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766433.803218:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766433/real 1775766433]  req@ffff9f14ab4ff100 x1862022114502784/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766458 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766433.803226:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff100 x1862022114502784/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766458 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766433.803231:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766433.803233:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766433.803234:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114502784:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775766433.803238:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114502912:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775766433.803247:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766433.803252:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766433/real 1775766433]  req@ffff9f14ab4fdc00 x1862022114502912/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766458 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766433.803259:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdc00 x1862022114502912/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766458 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766433.803268:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766433.803270:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766433.803272:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114502912:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775766433.803275:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114503040:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775766433.803284:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766433.803289:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766433/real 1775766433]  req@ffff9f14ab4fed80 x1862022114503040/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766458 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766433.803296:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fed80 x1862022114503040/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766458 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766433.803300:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766433.803302:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766433.803304:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114503040:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766433.803334:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114503168:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775766433.803353:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766433.803361:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766433/real 1775766433]  req@ffff9f14ab4fca80 x1862022114503168/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766458 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766433.803370:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fca80 x1862022114503168/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766458 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766433.803376:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766433.803378:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766433.803381:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114503168:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766433.803385:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114503296:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775766433.803397:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766433.803403:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766433/real 1775766433]  req@ffff9f14ab4fce00 x1862022114503296/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766458 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766433.803410:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fce00 x1862022114503296/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766458 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766433.803416:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766433.803418:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766433.803420:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114503296:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775766434.058730:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766434.826629:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766435.082722:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766435.850627:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766436.106812:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766436.874743:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766437.130657:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766437.898638:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775766438.154837:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1775766438.922659:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:2.0:1775766438.922664:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:1.0:1775766438.922667:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1775766438.922673:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766438.922681:0:8:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3816
00000100:00080000:1.0:1775766438.922691:0:8:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766438.922697:0:8:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775766438.922732:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fea00 x1862022114503424/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766438.922781:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766438.922784:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775766438.922786:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766438.922789:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3816
00000100:00080000:1.0:1775766438.922792:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766438.922795:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766438.922804:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775766438.922816:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff800 x1862022114503552/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766438.922824:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766438.922828:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00100000:3.0:1775766438.922829:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114503424:192.168.202.110@tcp:250:kworker.0
00000100:00080000:1.0:1775766438.922830:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766438.922838:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3816
00000100:00080000:1.0:1775766438.922841:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766438.922843:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766438.922846:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775766438.922854:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdf80 x1862022114503680/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766438.922860:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766438.922862:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775766438.922864:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766438.922866:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3816
00000100:00080000:1.0:1775766438.922868:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766438.922870:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766438.922873:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775766438.922879:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ffb80 x1862022114503808/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766438.922889:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766438.922892:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1775766438.922893:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766438.922895:0:8:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3816
00000100:00080000:1.0:1775766438.922897:0:8:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766438.922899:0:8:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766438.922902:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:3.0:1775766438.922906:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775766438.922911:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fce00 x1862022114503936/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766438.922917:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766438.922919:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775766438.922920:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766438.922923:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3816
00000100:00080000:1.0:1775766438.922928:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766438.922931:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766438.922934:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775766438.922941:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fca80 x1862022114504064/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766438.922947:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766438.922949:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775766438.922950:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:3.0:1775766438.922954:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766438/real 1775766438]  req@ffff9f14ab4fea00 x1862022114503424/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766463 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766438.922956:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3816
00000100:00080000:1.0:1775766438.922961:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766438.922967:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766438.922974:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775766438.922983:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fed80 x1862022114504192/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775766438.922989:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fea00 x1862022114503424/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766463 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766438.922991:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766438.922998:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775766438.923000:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766438.923008:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3816
00000100:00080000:1.0:1775766438.923012:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766438.923014:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00080000:3.0:1775766438.923014:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000008:00100000:1.0:1775766438.923021:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00080000:3.0:1775766438.923025:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775766438.923030:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdc00 x1862022114504320/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775766438.923032:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114503424:192.168.202.110@tcp:250:kworker.0
00000100:00100000:3.0:1775766438.923065:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114503552:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775766438.923092:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766438.923101:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766438/real 1775766438]  req@ffff9f14ab4ff800 x1862022114503552/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766463 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766438.923110:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff800 x1862022114503552/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766463 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766438.923116:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766438.923118:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766438.923121:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114503552:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766438.923125:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114503680:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775766438.923135:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766438.923141:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766438/real 1775766438]  req@ffff9f14ab4fdf80 x1862022114503680/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766463 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766438.923148:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdf80 x1862022114503680/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766463 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766438.923153:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766438.923154:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766438.923156:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114503680:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766438.923161:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ffb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114503808:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775766438.923178:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766438.923188:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766438/real 1775766438]  req@ffff9f14ab4ffb80 x1862022114503808/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766463 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766438.923207:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ffb80 x1862022114503808/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766463 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766438.923216:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766438.923219:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766438.923223:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ffb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114503808:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775766438.923229:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114503936:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775766438.923243:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766438.923250:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766438/real 1775766438]  req@ffff9f14ab4fce00 x1862022114503936/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766463 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766438.923258:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fce00 x1862022114503936/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766463 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766438.923263:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766438.923265:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766438.923267:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114503936:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775766438.923270:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114504064:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775766438.923279:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766438.923284:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766438/real 1775766438]  req@ffff9f14ab4fca80 x1862022114504064/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766463 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766438.923291:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fca80 x1862022114504064/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766463 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766438.923300:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766438.923302:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766438.923304:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114504064:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766438.923307:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114504192:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775766438.923316:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766438.923321:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766438/real 1775766438]  req@ffff9f14ab4fed80 x1862022114504192/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766463 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766438.923328:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fed80 x1862022114504192/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766463 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766438.923333:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766438.923334:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766438.923336:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114504192:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766438.923339:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114504320:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775766438.923348:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766438.923353:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766438/real 1775766438]  req@ffff9f14ab4fdc00 x1862022114504320/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766463 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766438.923360:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdc00 x1862022114504320/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766463 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766438.923365:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766438.923366:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766438.923369:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114504320:192.168.202.110@tcp:8:kworker.0
00010000:00010000:3.0:1775766439.178685:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766439.946631:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766440.202696:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766440.970660:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766441.226738:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766441.994681:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766442.250732:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766443.018652:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766443.274740:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766444.042665:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775766444.042812:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766444.042819:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775766444.042824:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766444.042834:0:8:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3821
00000100:00080000:3.0:1775766444.042846:0:8:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766444.042854:0:8:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775766444.042890:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746680 x1862022114504448/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766444.042918:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766444.042922:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775766444.042925:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766444.042931:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3821
00000100:00080000:3.0:1775766444.042935:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766444.042939:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775766444.042949:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114504448:192.168.202.110@tcp:250:kworker.0
00000008:00100000:3.0:1775766444.042950:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766444.042971:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744700 x1862022114504576/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766444.042985:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766444.042989:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775766444.042991:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766444.042995:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3821
00000100:00080000:3.0:1775766444.043009:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766444.043015:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766444.043021:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000800:00000100:2.0:1775766444.043023:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766444.043039:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745500 x1862022114504704/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766444.043048:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766444.043052:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775766444.043054:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766444.043060:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3821
00000100:00080000:3.0:1775766444.043064:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766444.043068:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775766444.043073:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766444/real 1775766444]  req@ffff9f1485746680 x1862022114504448/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766469 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:3.0:1775766444.043073:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775766444.043095:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746680 x1862022114504448/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766469 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766444.043106:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766444.043111:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766444.043114:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114504448:192.168.202.110@tcp:250:kworker.0
00000100:00100000:2.0:1775766444.043134:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114504576:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775766444.043154:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766444.043164:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766444/real 1775766444]  req@ffff9f1485744700 x1862022114504576/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766469 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766444.043187:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744700 x1862022114504576/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766469 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766444.043195:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766444.043198:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766444.043201:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114504576:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766444.043208:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114504704:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775766444.043224:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766444.043240:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766444/real 1775766444]  req@ffff9f1485745500 x1862022114504704/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766469 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766444.043253:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745500 x1862022114504704/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766469 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766444.043260:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766444.043262:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766444.043265:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114504704:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766444.043431:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744000 x1862022114504832/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766444.043449:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766444.043453:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775766444.043455:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766444.043461:0:8:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3821
00000100:00080000:3.0:1775766444.043466:0:8:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766444.043470:0:8:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775766444.043477:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114504832:192.168.202.110@tcp:8:kworker.0
00000008:00100000:3.0:1775766444.043477:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766444.043494:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747800 x1862022114504960/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766444.043505:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766444.043508:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775766444.043510:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766444.043514:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3821
00000100:00080000:3.0:1775766444.043518:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000800:00000100:2.0:1775766444.043524:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775766444.043525:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766444.043538:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766444.043550:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745c00 x1862022114505088/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766444.043596:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:2.0:1775766444.043599:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766444/real 1775766444]  req@ffff9f1485744000 x1862022114504832/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766469 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766444.043601:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775766444.043604:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766444.043616:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3821
00000100:00080000:3.0:1775766444.043624:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766444.043630:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766444.043636:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766444.043640:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744000 x1862022114504832/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766469 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766444.043656:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00100000:3.0:1775766444.043657:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745180 x1862022114505216/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766444.043668:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766444.043701:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114504832:192.168.202.110@tcp:8:kworker.0
00000100:00080000:3.0:1775766444.043702:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766444.043715:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775766444.043718:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766444.043732:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3821
00000100:00080000:3.0:1775766444.043738:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:2.0:1775766444.043742:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114504960:192.168.202.110@tcp:8:kworker.0
00000100:00080000:3.0:1775766444.043749:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766444.043757:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766444.043774:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745f80 x1862022114505344/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:2.0:1775766444.043788:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766444.043810:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766444/real 1775766444]  req@ffff9f1485747800 x1862022114504960/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766469 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766444.043829:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747800 x1862022114504960/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766469 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766444.043842:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766444.043847:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766444.043852:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114504960:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775766444.043862:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114505088:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775766444.043891:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766444.043907:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766444/real 1775766444]  req@ffff9f1485745c00 x1862022114505088/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766469 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766444.043925:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745c00 x1862022114505088/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766469 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766444.043938:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766444.043943:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766444.043949:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114505088:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766444.043957:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114505216:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775766444.043987:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766444.044002:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766444/real 1775766444]  req@ffff9f1485745180 x1862022114505216/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766469 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766444.044021:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745180 x1862022114505216/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766469 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766444.044043:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766444.044060:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766444.044066:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114505216:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766444.044109:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114505344:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775766444.044165:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766444.044181:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766444/real 1775766444]  req@ffff9f1485745f80 x1862022114505344/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766469 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766444.044200:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745f80 x1862022114505344/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766469 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766444.044213:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766444.044217:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766444.044222:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114505344:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775766444.298740:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766445.066642:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775766445.322793:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766446.090655:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775766446.346712:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766447.114660:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766447.370752:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766448.138620:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766448.394693:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766449.162670:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775766449.162686:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766449.162694:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775766449.162699:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766449.162710:0:8:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3826
00000100:00080000:3.0:1775766449.162721:0:8:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766449.162727:0:8:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775766449.162762:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747100 x1862022114505472/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766449.162788:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766449.162791:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775766449.162792:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766449.162795:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3826
00000100:00080000:3.0:1775766449.162798:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766449.162800:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766449.162808:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766449.162819:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745f80 x1862022114505600/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766449.162826:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766449.162829:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775766449.162831:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766449.162835:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3826
00000100:00080000:3.0:1775766449.162838:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766449.162843:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775766449.162845:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114505472:192.168.202.110@tcp:250:kworker.0
00000008:00100000:3.0:1775766449.162847:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766449.162858:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745180 x1862022114505728/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766449.162866:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766449.162868:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775766449.162870:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766449.162873:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3826
00000100:00080000:3.0:1775766449.162875:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766449.162878:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766449.162882:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766449.162890:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745c00 x1862022114505856/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766449.162898:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766449.162900:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775766449.162902:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766449.162904:0:8:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3826
00000100:00080000:3.0:1775766449.162907:0:8:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000800:00000100:2.0:1775766449.162912:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775766449.162914:0:8:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766449.162919:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766449.162927:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747800 x1862022114505984/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766449.162934:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766449.162937:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775766449.162938:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766449.162942:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3826
00000100:00080000:3.0:1775766449.162944:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766449.162948:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766449.162952:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766449.162961:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744000 x1862022114506112/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775766449.162964:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766449/real 1775766449]  req@ffff9f1485747100 x1862022114505472/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766474 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766449.162977:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766449.162985:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775766449.162989:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766449.162993:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3826
00000100:00080000:3.0:1775766449.162996:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:2.0:1775766449.163000:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747100 x1862022114505472/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766474 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766449.163000:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766449.163007:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00080000:2.0:1775766449.163016:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766449.163021:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766449.163025:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114505472:192.168.202.110@tcp:250:kworker.0
00000100:00100000:2.0:1775766449.163034:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114505600:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766449.163034:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745500 x1862022114506240/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766449.163048:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766449.163051:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775766449.163052:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766449.163056:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3826
00000100:00080000:3.0:1775766449.163059:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766449.163074:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766449.163079:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:2.0:1775766449.163084:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766449.163088:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744700 x1862022114506368/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775766449.163104:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766449/real 1775766449]  req@ffff9f1485745f80 x1862022114505600/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766474 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766449.163121:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745f80 x1862022114505600/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766474 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766449.163141:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766449.163144:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766449.163147:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114505600:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766449.163167:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114505728:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775766449.163194:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766449.163203:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766449/real 1775766449]  req@ffff9f1485745180 x1862022114505728/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766474 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766449.163212:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745180 x1862022114505728/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766474 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766449.163222:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766449.163225:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766449.163227:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114505728:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766449.163231:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114505856:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775766449.163244:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766449.163250:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766449/real 1775766449]  req@ffff9f1485745c00 x1862022114505856/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766474 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766449.163265:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745c00 x1862022114505856/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766474 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766449.163271:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766449.163273:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766449.163275:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114505856:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775766449.163279:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114505984:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775766449.163290:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766449.163296:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766449/real 1775766449]  req@ffff9f1485747800 x1862022114505984/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766474 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766449.163304:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747800 x1862022114505984/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766474 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766449.163317:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766449.163319:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766449.163322:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114505984:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775766449.163326:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114506112:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775766449.163338:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766449.163344:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766449/real 1775766449]  req@ffff9f1485744000 x1862022114506112/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766474 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766449.163353:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744000 x1862022114506112/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766474 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766449.163358:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766449.163360:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766449.163362:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114506112:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766449.163366:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114506240:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775766449.163376:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766449.163382:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766449/real 1775766449]  req@ffff9f1485745500 x1862022114506240/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766474 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766449.163390:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745500 x1862022114506240/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766474 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766449.163394:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766449.163396:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766449.163398:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114506240:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766449.163402:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114506368:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775766449.163412:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766449.163418:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766449/real 1775766449]  req@ffff9f1485744700 x1862022114506368/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766474 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766449.163425:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744700 x1862022114506368/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766474 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766449.163431:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766449.163432:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766449.163434:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114506368:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775766449.418715:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766450.186661:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766450.442769:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766451.210673:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766451.466675:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766452.234698:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775766452.490775:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766453.258745:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775766453.514786:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1775766454.282721:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766454.282736:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1775766454.282743:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775766454.282755:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:2.0:1775766454.282760:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3831
00000100:00080000:2.0:1775766454.282781:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766454.282792:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775766454.282856:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1c00 x1862022114506496/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766454.282895:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766454.282900:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775766454.282903:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766454.282909:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3831
00000100:00080000:2.0:1775766454.282914:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766454.282920:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766454.282946:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766454.282964:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3480 x1862022114506624/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775766454.282975:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114506496:192.168.202.110@tcp:250:kworker.0
00000100:00080000:2.0:1775766454.282986:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766454.282992:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775766454.282994:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766454.283000:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3831
00000100:00080000:2.0:1775766454.283009:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766454.283013:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766454.283020:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766454.283038:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2300 x1862022114506752/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766454.283050:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766454.283054:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775766454.283056:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766454.283060:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3831
00000100:00080000:2.0:1775766454.283077:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766454.283083:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766454.283090:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:3.0:1775766454.283102:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766454.283109:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3100 x1862022114506880/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766454.283123:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766454.283128:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1775766454.283132:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766454.283138:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3831
00000100:00080000:2.0:1775766454.283142:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766454.283146:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766454.283152:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775766454.283165:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1880 x1862022114507008/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766454.283177:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:3.0:1775766454.283180:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766454/real 1775766454]  req@ffff9f14852d1c00 x1862022114506496/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766479 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766454.283186:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775766454.283193:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766454.283208:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3831
00000100:00080000:2.0:1775766454.283216:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766454.283224:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766454.283230:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766454.283246:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1c00 x1862022114506496/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766479 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766454.283248:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0e00 x1862022114507136/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766454.283293:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766454.283303:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775766454.283306:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766454.283311:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766454.283312:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3831
00000100:00080000:2.0:1775766454.283322:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766454.283331:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00080000:3.0:1775766454.283331:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000008:00100000:2.0:1775766454.283345:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766454.283349:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114506496:192.168.202.110@tcp:250:kworker.0
00000100:00100000:2.0:1775766454.283361:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3b80 x1862022114507264/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766454.283373:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766454.283378:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775766454.283381:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766454.283386:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3831
00000100:00080000:2.0:1775766454.283390:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766454.283394:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766454.283401:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766454.283407:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114506624:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766454.283418:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0000 x1862022114507392/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:3.0:1775766454.283478:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766454.283500:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766454/real 1775766454]  req@ffff9f14852d3480 x1862022114506624/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766479 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766454.283518:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3480 x1862022114506624/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766479 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766454.283539:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766454.283543:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766454.283548:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114506624:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766454.283622:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114506752:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775766454.283656:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766454.283672:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766454/real 1775766454]  req@ffff9f14852d2300 x1862022114506752/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766479 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766454.283688:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2300 x1862022114506752/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766479 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766454.283698:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766454.283703:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766454.283707:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114506752:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766454.283715:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114506880:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775766454.283747:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766454.283767:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766454/real 1775766454]  req@ffff9f14852d3100 x1862022114506880/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766479 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766454.283792:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3100 x1862022114506880/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766479 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766454.283809:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766454.283816:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766454.283824:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114506880:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775766454.283836:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114507008:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775766454.283873:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766454.283889:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766454/real 1775766454]  req@ffff9f14852d1880 x1862022114507008/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766479 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766454.283904:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1880 x1862022114507008/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766479 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766454.283914:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766454.283918:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766454.283922:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114507008:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775766454.283929:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114507136:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775766454.283950:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766454.283960:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766454/real 1775766454]  req@ffff9f14852d0e00 x1862022114507136/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766479 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766454.283975:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0e00 x1862022114507136/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766479 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766454.283984:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766454.283987:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766454.283990:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114507136:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766454.283996:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114507264:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775766454.284015:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766454.284027:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766454/real 1775766454]  req@ffff9f14852d3b80 x1862022114507264/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766479 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766454.284041:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3b80 x1862022114507264/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766479 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766454.284051:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766454.284054:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766454.284057:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114507264:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766454.284087:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114507392:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775766454.284106:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766454.284117:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766454/real 1775766454]  req@ffff9f14852d0000 x1862022114507392/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766479 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766454.284132:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0000 x1862022114507392/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766479 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766454.284141:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766454.284145:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766454.284148:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114507392:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775766454.538864:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766455.306732:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766455.562929:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766456.330642:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766456.586839:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766457.354662:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766457.610699:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766458.378692:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766458.634692:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766459.402685:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775766459.402687:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766459.402697:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775766459.402702:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766459.402716:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3836
00000100:00080000:3.0:1775766459.402730:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766459.402740:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775766459.402796:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746680 x1862022114507520/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766459.402832:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766459.402837:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775766459.402840:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766459.402846:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3836
00000100:00080000:3.0:1775766459.402853:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766459.402860:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775766459.402869:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114507520:192.168.202.110@tcp:250:kworker.0
00000008:00100000:3.0:1775766459.402873:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766459.402894:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747b80 x1862022114507648/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766459.402911:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766459.402916:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775766459.402918:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766459.402925:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3836
00000100:00080000:3.0:1775766459.402931:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000800:00000100:2.0:1775766459.402937:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775766459.402940:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766459.402950:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766459.402968:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744e00 x1862022114507776/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775766459.402980:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766459/real 1775766459]  req@ffff9f1485746680 x1862022114507520/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766484 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766459.402984:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766459.402996:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775766459.403002:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766459.403016:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3836
00000100:00100000:2.0:1775766459.403021:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746680 x1862022114507520/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766484 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766459.403023:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766459.403035:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00080000:2.0:1775766459.403039:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000008:00100000:3.0:1775766459.403044:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00080000:2.0:1775766459.403046:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766459.403053:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114507520:192.168.202.110@tcp:250:kworker.0
00000100:00100000:3.0:1775766459.403065:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744380 x1862022114507904/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775766459.403076:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114507648:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775766459.403084:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766459.403090:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775766459.403093:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000800:00000100:2.0:1775766459.403102:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775766459.403104:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3836
00000100:00080000:3.0:1775766459.403113:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00100000:2.0:1775766459.403118:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766459/real 1775766459]  req@ffff9f1485747b80 x1862022114507648/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766484 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766459.403123:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775766459.403140:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747b80 x1862022114507648/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766484 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766459.403147:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000008:00100000:3.0:1775766459.403150:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00080000:2.0:1775766459.403151:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766459.403157:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114507648:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766459.403165:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114507776:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766459.403174:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746a00 x1862022114508032/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:2.0:1775766459.403185:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775766459.403196:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:2.0:1775766459.403201:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766459/real 1775766459]  req@ffff9f1485744e00 x1862022114507776/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766484 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766459.403203:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775766459.403208:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766459.403218:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3836
00000100:00080000:3.0:1775766459.403228:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:2.0:1775766459.403233:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744e00 x1862022114507776/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766484 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766459.403239:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766459.403252:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00080000:2.0:1775766459.403268:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00100000:3.0:1775766459.403275:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746300 x1862022114508160/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766459.403277:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766459.403285:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114507776:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775766459.403303:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:2.0:1775766459.403309:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114507904:192.168.202.110@tcp:8:kworker.0
00000100:00080000:3.0:1775766459.403309:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775766459.403315:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766459.403323:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3836
00000800:00000100:2.0:1775766459.403337:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775766459.403339:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766459.403347:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775766459.403350:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766459/real 1775766459]  req@ffff9f1485744380 x1862022114507904/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766484 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:3.0:1775766459.403359:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766459.403365:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744380 x1862022114507904/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766484 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766459.403373:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766459.403377:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766459.403382:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747480 x1862022114508288/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775766459.403384:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114507904:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775766459.403396:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114508032:192.168.202.110@tcp:8:kworker.0
00000100:00080000:3.0:1775766459.403413:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766459.403418:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000800:00000100:2.0:1775766459.403421:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775766459.403423:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766459.403431:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3836
00000100:00100000:2.0:1775766459.403435:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766459/real 1775766459]  req@ffff9f1485746a00 x1862022114508032/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766484 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766459.403439:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766459.403452:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775766459.403454:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746a00 x1862022114508032/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766484 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:3.0:1775766459.403464:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00080000:2.0:1775766459.403465:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766459.403473:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766459.403476:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114508032:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775766459.403488:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114508160:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766459.403489:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745880 x1862022114508416/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:2.0:1775766459.403508:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766459.403548:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766459/real 1775766459]  req@ffff9f1485746300 x1862022114508160/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766484 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766459.403581:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746300 x1862022114508160/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766484 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766459.403599:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766459.403602:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766459.403605:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114508160:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766459.403618:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114508288:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775766459.403636:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766459.403645:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766459/real 1775766459]  req@ffff9f1485747480 x1862022114508288/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766484 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766459.403655:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747480 x1862022114508288/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766484 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766459.403661:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766459.403664:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766459.403667:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114508288:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766459.403671:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114508416:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775766459.403682:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766459.403689:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766459/real 1775766459]  req@ffff9f1485745880 x1862022114508416/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766484 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766459.403698:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745880 x1862022114508416/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766484 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766459.403703:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766459.403706:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766459.403708:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114508416:192.168.202.110@tcp:8:kworker.0
00010000:00010000:3.0:1775766459.658701:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766460.426658:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775766460.682688:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766461.450610:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766461.706687:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766462.474630:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766462.730681:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766463.498647:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766463.754825:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766464.522647:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775766464.522647:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766464.522656:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775766464.522660:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766464.522668:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3842
00000100:00080000:3.0:1775766464.522677:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766464.522683:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775766464.522715:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744a80 x1862022114508544/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766464.522735:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766464.522738:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775766464.522739:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766464.522742:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3842
00000100:00080000:3.0:1775766464.522745:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766464.522747:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766464.522755:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766464.522765:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114508544:192.168.202.110@tcp:250:kworker.0
00000100:00100000:3.0:1775766464.522771:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745880 x1862022114508672/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766464.522785:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766464.522789:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775766464.522791:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766464.522796:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3842
00000100:00080000:3.0:1775766464.522809:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766464.522813:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766464.522817:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766464.522827:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747480 x1862022114508800/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766464.522840:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766464.522844:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775766464.522846:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766464.522851:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3842
00000100:00080000:3.0:1775766464.522855:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00100000:2.0:1775766464.522860:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114508672:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775766464.522861:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766464.522870:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766464.522885:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746300 x1862022114508928/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775766464.522889:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114508800:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775766464.522905:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766464.522907:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775766464.522909:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:2.0:1775766464.522912:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114508928:192.168.202.110@tcp:8:kworker.0
00000100:00080000:3.0:1775766464.522927:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3842
00000100:00080000:3.0:1775766464.522933:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766464.522937:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766464.522944:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766464.522957:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746a00 x1862022114509056/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766464.522976:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766464.522980:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775766464.522982:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766464.522987:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3842
00000100:00080000:3.0:1775766464.522991:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766464.522995:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775766464.523000:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114509056:192.168.202.110@tcp:8:kworker.0
00000008:00100000:3.0:1775766464.523000:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766464.523017:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744380 x1862022114509184/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766464.523025:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766464.523028:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775766464.523030:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:2.0:1775766464.523032:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114509184:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775766464.523036:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3842
00000100:00080000:3.0:1775766464.523041:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766464.523044:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766464.523048:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766464.523058:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744e00 x1862022114509312/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766464.523068:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766464.523070:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775766464.523072:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766464.523075:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3842
00000100:00100000:2.0:1775766464.523082:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114509312:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775766464.523089:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766464.523096:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766464.523102:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766464.523117:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747b80 x1862022114509440/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775766464.523161:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114509440:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775766464.778727:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766465.546620:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766465.802709:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766466.570648:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775766466.826749:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766467.594644:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000400:00000100:1.0:1775766467.594955:0:2390:0:(lib-socket.c:454:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.202.110:988
00000400:00000100:1.0:1775766467.595012:0:2390:0:(acceptor.c:111:lnet_connect_console_error()) Connection to 192.168.202.110@tcp at host 192.168.202.110 was unreachable: the network or that node may be down, or Lustre may be misconfigured.
00000800:00000100:1.0:1775766467.595033:0:2390:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:1.0:1775766467.595088:0:2390:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:1.0:1775766467.595099:0:2390:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:1.0:1775766467.595108:0:2390:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:1.0:1775766467.595117:0:2390:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:1.0:1775766467.595126:0:2390:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:1.0:1775766467.595135:0:2390:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:1.0:1775766467.595147:0:2390:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000100:00100000:2.0:1775766467.595178:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766464/real 1775766467]  req@ffff9f1485747b80 x1862022114509440/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766489 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766467.595212:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747b80 x1862022114509440/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766489 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766467.595225:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766467.595231:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766467.595237:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114509440:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775766467.595247:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766464/real 1775766467]  req@ffff9f1485744e00 x1862022114509312/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766489 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766467.595261:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744e00 x1862022114509312/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766489 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766467.595271:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766467.595275:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766467.595279:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114509312:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766467.595287:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766464/real 1775766467]  req@ffff9f1485744380 x1862022114509184/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766489 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766467.595308:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744380 x1862022114509184/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766489 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766467.595317:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766467.595321:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766467.595325:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114509184:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766467.595334:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766464/real 1775766467]  req@ffff9f1485746a00 x1862022114509056/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766489 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766467.595349:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746a00 x1862022114509056/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766489 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766467.595357:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766467.595362:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766467.595365:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114509056:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775766467.595373:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766464/real 1775766467]  req@ffff9f1485746300 x1862022114508928/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766489 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766467.595385:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746300 x1862022114508928/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766489 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766467.595394:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766467.595398:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766467.595402:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114508928:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775766467.595410:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766464/real 1775766467]  req@ffff9f1485745880 x1862022114508672/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766489 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766467.595424:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745880 x1862022114508672/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766489 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766467.595464:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766467.595470:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766467.595474:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114508672:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766467.595482:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766464/real 1775766467]  req@ffff9f1485747480 x1862022114508800/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766489 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766467.595496:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747480 x1862022114508800/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766489 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766467.595504:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766467.595508:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766467.595511:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114508800:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766467.595518:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766464/real 1775766467]  req@ffff9f1485744a80 x1862022114508544/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766489 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766467.595531:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744a80 x1862022114508544/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766489 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766467.595540:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766467.595544:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766467.595548:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114508544:192.168.202.110@tcp:250:kworker.0
00010000:00010000:3.0:1775766467.850719:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766468.618674:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766468.874753:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1775766469.642649:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:2.0:1775766469.642654:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775766469.642662:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775766469.642667:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766469.642677:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3847
00000100:00080000:3.0:1775766469.642691:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766469.642699:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775766469.642739:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746680 x1862022114509568/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766469.642767:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766469.642770:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775766469.642772:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766469.642775:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3847
00000100:00080000:3.0:1775766469.642778:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766469.642780:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766469.642791:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766469.642800:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744700 x1862022114509696/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766469.642807:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766469.642810:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775766469.642812:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766469.642815:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3847
00000100:00080000:3.0:1775766469.642818:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766469.642823:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775766469.642825:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114509568:192.168.202.110@tcp:250:kworker.0
00000008:00100000:3.0:1775766469.642827:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766469.642838:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745500 x1862022114509824/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766469.642845:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766469.642848:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775766469.642849:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766469.642852:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3847
00000100:00080000:3.0:1775766469.642854:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766469.642856:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766469.642860:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766469.642873:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744000 x1862022114509952/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766469.642884:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766469.642888:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775766469.642889:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766469.642893:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3847
00000100:00080000:3.0:1775766469.642910:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766469.642916:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766469.642923:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766469.642939:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747800 x1862022114510080/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:2.0:1775766469.642943:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775766469.642961:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766469.642966:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775766469.642968:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766469.642975:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3847
00000100:00080000:3.0:1775766469.642980:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766469.642986:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766469.642991:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766469.643006:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745c00 x1862022114510208/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775766469.643017:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766469/real 1775766469]  req@ffff9f1485746680 x1862022114509568/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766494 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766469.643018:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766469.643027:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775766469.643031:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766469.643040:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3847
00000100:00080000:3.0:1775766469.643046:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766469.643050:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775766469.643057:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746680 x1862022114509568/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766494 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:3.0:1775766469.643057:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766469.643074:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745180 x1862022114510336/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766469.643076:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766469.643087:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:3.0:1775766469.643092:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:2.0:1775766469.643096:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114509568:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775766469.643098:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775766469.643102:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766469.643111:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3847
00000100:00080000:3.0:1775766469.643116:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:2.0:1775766469.643117:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114509696:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775766469.643125:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766469.643130:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766469.643144:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745f80 x1862022114510464/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:2.0:1775766469.643160:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766469.643178:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766469/real 1775766469]  req@ffff9f1485744700 x1862022114509696/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766494 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766469.643193:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744700 x1862022114509696/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766494 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766469.643202:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766469.643206:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766469.643210:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114509696:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766469.643250:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114509824:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775766469.643274:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766469.643287:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766469/real 1775766469]  req@ffff9f1485745500 x1862022114509824/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766494 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766469.643300:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745500 x1862022114509824/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766494 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766469.643310:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766469.643314:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766469.643317:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114509824:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766469.643324:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114509952:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775766469.643349:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766469.643361:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766469/real 1775766469]  req@ffff9f1485744000 x1862022114509952/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766494 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766469.643374:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744000 x1862022114509952/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766494 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766469.643384:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766469.643389:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766469.643392:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114509952:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775766469.643397:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114510080:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775766469.643455:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766469.643468:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766469/real 1775766469]  req@ffff9f1485747800 x1862022114510080/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766494 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766469.643480:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747800 x1862022114510080/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766494 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766469.643489:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766469.643493:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766469.643496:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114510080:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775766469.643502:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114510208:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775766469.643524:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766469.643536:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766469/real 1775766469]  req@ffff9f1485745c00 x1862022114510208/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766494 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766469.643547:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745c00 x1862022114510208/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766494 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766469.643598:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766469.643603:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766469.643606:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114510208:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766469.643614:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114510336:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775766469.643645:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766469.643658:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766469/real 1775766469]  req@ffff9f1485745180 x1862022114510336/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766494 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766469.643675:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745180 x1862022114510336/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766494 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766469.643684:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766469.643688:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766469.643692:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114510336:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766469.643700:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114510464:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775766469.643723:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766469.643732:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766469/real 1775766469]  req@ffff9f1485745f80 x1862022114510464/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766494 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766469.643743:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745f80 x1862022114510464/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766494 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766469.643751:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766469.643754:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766469.643757:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114510464:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775766469.898755:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766470.666679:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766470.922782:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766471.690632:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766471.946742:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766472.714678:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766472.970720:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766473.738635:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775766473.994749:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766474.762766:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775766474.762770:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766474.762784:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775766474.762791:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766474.762806:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3852
00000100:00080000:3.0:1775766474.762824:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766474.762846:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775766474.762914:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747100 x1862022114510592/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766474.762951:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766474.762956:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775766474.762959:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766474.762964:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3852
00000100:00080000:3.0:1775766474.762969:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766474.762974:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766474.762989:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766474.763008:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745f80 x1862022114510720/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775766474.763016:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114510592:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775766474.763041:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766474.763046:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775766474.763048:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766474.763054:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3852
00000100:00080000:3.0:1775766474.763059:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766474.763064:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766474.763070:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766474.763087:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745180 x1862022114510848/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766474.763099:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766474.763102:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775766474.763104:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766474.763110:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3852
00000100:00080000:3.0:1775766474.763117:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766474.763122:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766474.763127:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:2.0:1775766474.763138:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766474.763145:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745c00 x1862022114510976/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766474.763157:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766474.763161:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775766474.763163:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766474.763182:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3852
00000100:00080000:3.0:1775766474.763189:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766474.763193:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766474.763201:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766474.763216:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747800 x1862022114511104/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775766474.763232:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766474/real 1775766474]  req@ffff9f1485747100 x1862022114510592/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766499 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766474.763238:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766474.763255:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775766474.763260:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766474.763281:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3852
00000100:00080000:3.0:1775766474.763286:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:2.0:1775766474.763293:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747100 x1862022114510592/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766499 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766474.763294:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766474.763307:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00080000:2.0:1775766474.763327:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00100000:3.0:1775766474.763335:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744000 x1862022114511232/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766474.763342:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766474.763393:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114510592:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775766474.763393:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766474.763409:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775766474.763414:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766474.763422:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3852
00000100:00080000:3.0:1775766474.763428:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766474.763433:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766474.763442:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766474.763447:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114510720:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766474.763465:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745500 x1862022114511360/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766474.763482:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766474.763487:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775766474.763491:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000800:00000100:2.0:1775766474.763500:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775766474.763506:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3852
00000100:00080000:3.0:1775766474.763514:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766474.763520:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766474.763528:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775766474.763534:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766474/real 1775766474]  req@ffff9f1485745f80 x1862022114510720/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766499 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766474.763615:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744700 x1862022114511488/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775766474.763618:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745f80 x1862022114510720/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766499 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766474.763677:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766474.763686:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766474.763696:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114510720:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766474.763711:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114510848:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775766474.763749:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766474.763766:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766474/real 1775766474]  req@ffff9f1485745180 x1862022114510848/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766499 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766474.763782:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745180 x1862022114510848/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766499 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766474.763792:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766474.763796:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766474.763800:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114510848:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766474.763809:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114510976:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775766474.763829:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766474.763848:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766474/real 1775766474]  req@ffff9f1485745c00 x1862022114510976/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766499 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766474.763863:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745c00 x1862022114510976/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766499 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766474.763872:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766474.763876:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766474.763881:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114510976:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775766474.763892:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114511104:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775766474.763929:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766474.763952:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766474/real 1775766474]  req@ffff9f1485747800 x1862022114511104/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766499 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766474.763976:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747800 x1862022114511104/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766499 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766474.763993:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766474.763999:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766474.764006:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114511104:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775766474.764018:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114511232:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775766474.764054:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766474.764070:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766474/real 1775766474]  req@ffff9f1485744000 x1862022114511232/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766499 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766474.764088:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744000 x1862022114511232/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766499 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766474.764105:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766474.764111:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766474.764117:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114511232:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766474.764157:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114511360:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775766474.764203:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766474.764229:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766474/real 1775766474]  req@ffff9f1485745500 x1862022114511360/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766499 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766474.764255:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745500 x1862022114511360/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766499 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766474.764272:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766474.764278:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766474.764284:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114511360:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766474.764298:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114511488:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775766474.764339:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766474.764388:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766474/real 1775766474]  req@ffff9f1485744700 x1862022114511488/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766499 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766474.764425:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744700 x1862022114511488/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766499 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766474.764454:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766474.764461:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766474.764467:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114511488:192.168.202.110@tcp:8:kworker.0
00010000:00010000:3.0:1775766475.018755:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766475.786773:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766476.042767:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766476.810694:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766477.066730:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766477.834699:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766478.090751:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766478.858713:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766479.114777:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766479.882686:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:0.0:1775766479.882710:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766479.882717:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1775766479.882721:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766479.882730:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3857
00000100:00080000:0.0:1775766479.882743:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766479.882750:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775766479.882784:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d1f80 x1862022114511616/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775766479.882807:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766479.882810:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775766479.882811:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766479.882814:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3857
00000100:00080000:0.0:1775766479.882817:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766479.882821:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775766479.882829:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775766479.882846:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0a80 x1862022114511744/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775766479.882846:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114511616:192.168.202.110@tcp:250:kworker.0
00000100:00080000:0.0:1775766479.882862:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766479.882872:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:0.0:1775766479.882875:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766479.882880:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3857
00000100:00080000:0.0:1775766479.882885:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766479.882889:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775766479.882896:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775766479.882908:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3b80 x1862022114511872/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775766479.882916:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766479.882921:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775766479.882922:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000800:00000100:3.0:1775766479.882922:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:0.0:1775766479.882931:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3857
00000100:00080000:0.0:1775766479.882936:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766479.882940:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775766479.882947:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775766479.882954:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d2a00 x1862022114512000/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775766479.882961:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766479.882964:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1775766479.882965:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766479.882967:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3857
00000100:00080000:0.0:1775766479.882970:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766479.882972:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775766479.882974:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775766479.882979:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3100 x1862022114512128/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775766479.883025:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766479.883027:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775766479.883029:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766479.883031:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3857
00000100:00080000:0.0:1775766479.883034:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766479.883037:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775766479.883040:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775766479.883051:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0380 x1862022114512256/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775766479.883061:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766479.883064:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:0.0:1775766479.883065:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766479.883071:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3857
00000100:00100000:3.0:1775766479.883071:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766479/real 1775766479]  req@ffff9f14a04d1f80 x1862022114511616/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766504 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766479.883075:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766479.883081:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775766479.883089:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775766479.883105:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0e00 x1862022114512384/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775766479.883111:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766479.883113:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775766479.883114:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766479.883117:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3857
00000100:00080000:0.0:1775766479.883119:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766479.883122:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775766479.883127:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775766479.883137:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d2300 x1862022114512512/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775766479.883219:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1f80 x1862022114511616/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766504 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766479.883233:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766479.883239:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766479.883244:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114511616:192.168.202.110@tcp:250:kworker.0
00000100:00100000:3.0:1775766479.883348:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114511744:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775766479.883374:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766479.883387:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766479/real 1775766479]  req@ffff9f14a04d0a80 x1862022114511744/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766504 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766479.883401:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0a80 x1862022114511744/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766504 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766479.883418:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766479.883422:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766479.883425:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114511744:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766479.883432:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114511872:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775766479.883449:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766479.883458:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766479/real 1775766479]  req@ffff9f14a04d3b80 x1862022114511872/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766504 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766479.883471:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3b80 x1862022114511872/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766504 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766479.883479:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766479.883483:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766479.883486:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114511872:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766479.883492:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114512000:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775766479.883507:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766479.883515:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766479/real 1775766479]  req@ffff9f14a04d2a00 x1862022114512000/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766504 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766479.883527:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d2a00 x1862022114512000/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766504 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766479.883535:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766479.883538:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766479.883542:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114512000:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775766479.883548:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114512128:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775766479.883604:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766479.883616:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766479/real 1775766479]  req@ffff9f14a04d3100 x1862022114512128/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766504 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766479.883628:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3100 x1862022114512128/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766504 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766479.883636:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766479.883640:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766479.883643:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114512128:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775766479.883650:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114512256:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775766479.883665:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766479.883674:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766479/real 1775766479]  req@ffff9f14a04d0380 x1862022114512256/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766504 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766479.883686:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0380 x1862022114512256/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766504 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766479.883694:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766479.883697:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766479.883701:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114512256:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766479.883706:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114512384:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775766479.883721:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766479.883730:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766479/real 1775766479]  req@ffff9f14a04d0e00 x1862022114512384/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766504 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766479.883741:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0e00 x1862022114512384/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766504 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766479.883749:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766479.883752:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766479.883755:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114512384:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766479.883761:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114512512:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775766479.883776:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766479.883784:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766479/real 1775766479]  req@ffff9f14a04d2300 x1862022114512512/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766504 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766479.883796:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d2300 x1862022114512512/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766504 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766479.883804:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766479.883807:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766479.883811:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114512512:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775766480.138719:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766480.906682:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775766481.162783:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766481.930665:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775766482.186725:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766482.954639:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766483.210768:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766483.978717:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766484.234712:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766485.002661:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775766485.002661:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766485.002670:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775766485.002674:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766485.002682:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3862
00000100:00080000:3.0:1775766485.002692:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766485.002698:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775766485.002730:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746680 x1862022114512640/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766485.002753:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766485.002756:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775766485.002757:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766485.002760:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3862
00000100:00080000:3.0:1775766485.002763:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766485.002765:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766485.002773:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766485.002782:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744a80 x1862022114512768/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775766485.002791:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114512640:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775766485.002792:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766485.002799:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775766485.002801:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766485.002805:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3862
00000100:00080000:3.0:1775766485.002807:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766485.002809:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766485.002812:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766485.002826:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747480 x1862022114512896/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766485.002833:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766485.002836:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775766485.002838:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766485.002840:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3862
00000100:00080000:3.0:1775766485.002848:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766485.002851:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766485.002854:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766485.002863:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745880 x1862022114513024/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:2.0:1775766485.002864:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775766485.002872:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766485.002875:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775766485.002876:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766485.002880:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3862
00000100:00080000:3.0:1775766485.002882:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766485.002885:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766485.002888:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766485.002894:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746300 x1862022114513152/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766485.002900:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766485.002903:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775766485.002905:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766485.002908:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3862
00000100:00080000:3.0:1775766485.002910:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:2.0:1775766485.002912:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766485/real 1775766485]  req@ffff9f1485746680 x1862022114512640/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766510 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766485.002915:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766485.002922:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766485.002936:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746a00 x1862022114513280/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775766485.002941:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746680 x1862022114512640/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766510 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766485.002950:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766485.002957:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775766485.002960:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766485.002967:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3862
00000100:00080000:3.0:1775766485.002972:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766485.002978:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766485.002985:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00080000:2.0:1775766485.002986:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00100000:3.0:1775766485.002993:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744380 x1862022114513408/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766485.002994:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766485.003003:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114512640:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775766485.003006:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766485.003010:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775766485.003012:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766485.003015:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3862
00000100:00080000:3.0:1775766485.003017:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766485.003020:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766485.003023:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775766485.003031:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114512768:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766485.003031:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744e00 x1862022114513536/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:2.0:1775766485.003061:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766485.003073:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766485/real 1775766485]  req@ffff9f1485744a80 x1862022114512768/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766510 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766485.003086:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744a80 x1862022114512768/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766510 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766485.003095:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766485.003099:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766485.003102:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114512768:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766485.003107:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114512896:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775766485.003121:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766485.003129:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766485/real 1775766485]  req@ffff9f1485747480 x1862022114512896/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766510 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766485.003140:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747480 x1862022114512896/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766510 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766485.003149:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766485.003151:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766485.003154:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114512896:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766485.003161:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114513024:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775766485.003175:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766485.003183:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766485/real 1775766485]  req@ffff9f1485745880 x1862022114513024/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766510 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766485.003194:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745880 x1862022114513024/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766510 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766485.003203:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766485.003206:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766485.003209:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114513024:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775766485.003214:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114513152:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775766485.003227:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766485.003235:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766485/real 1775766485]  req@ffff9f1485746300 x1862022114513152/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766510 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766485.003246:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746300 x1862022114513152/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766510 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766485.003287:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766485.003291:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766485.003294:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114513152:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775766485.003301:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114513280:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775766485.003318:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766485.003328:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766485/real 1775766485]  req@ffff9f1485746a00 x1862022114513280/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766510 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766485.003341:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746a00 x1862022114513280/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766510 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766485.003349:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766485.003352:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766485.003356:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114513280:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766485.003361:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114513408:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775766485.003375:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766485.003383:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766485/real 1775766485]  req@ffff9f1485744380 x1862022114513408/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766510 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766485.003394:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744380 x1862022114513408/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766510 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766485.003402:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766485.003405:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766485.003408:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114513408:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766485.003437:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114513536:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775766485.003456:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766485.003465:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766485/real 1775766485]  req@ffff9f1485744e00 x1862022114513536/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766510 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766485.003476:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744e00 x1862022114513536/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766510 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766485.003492:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766485.003495:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766485.003499:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114513536:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775766485.258696:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766486.026647:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766486.282795:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766487.050631:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766487.306791:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766488.074664:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775766488.330829:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766489.098625:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775766489.354733:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1775766490.122663:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:2.0:1775766490.122670:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775766490.122673:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775766490.122696:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766490.122707:0:8:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3867
00000100:00080000:3.0:1775766490.122718:0:8:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766490.122725:0:8:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775766490.122783:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747b80 x1862022114513664/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766490.122820:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766490.122826:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775766490.122828:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766490.122834:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3867
00000100:00080000:3.0:1775766490.122838:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766490.122843:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766490.122856:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766490.122875:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114513664:192.168.202.110@tcp:250:kworker.0
00000100:00100000:3.0:1775766490.122878:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744e00 x1862022114513792/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766490.122917:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766490.122920:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775766490.122922:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766490.122925:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3867
00000100:00080000:3.0:1775766490.122929:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766490.122931:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766490.122936:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766490.122946:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744380 x1862022114513920/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766490.122960:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766490.122963:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775766490.122965:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000800:00000100:2.0:1775766490.122970:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775766490.122973:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3867
00000100:00080000:3.0:1775766490.122977:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766490.122981:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766490.122986:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766490.122995:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746a00 x1862022114514048/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766490.123003:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766490.123006:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775766490.123008:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766490.123012:0:8:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3867
00000100:00100000:2.0:1775766490.123016:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766490/real 1775766490]  req@ffff9f1485747b80 x1862022114513664/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766515 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766490.123017:0:8:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766490.123026:0:8:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766490.123036:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775766490.123045:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747b80 x1862022114513664/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766515 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766490.123049:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746300 x1862022114514176/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766490.123072:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766490.123074:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766490.123079:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775766490.123080:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:3.0:1775766490.123084:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:2.0:1775766490.123089:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114513664:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775766490.123094:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3867
00000100:00080000:3.0:1775766490.123099:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766490.123103:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766490.123106:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766490.123115:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114513792:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766490.123126:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745880 x1862022114514304/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766490.123137:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000800:00000100:2.0:1775766490.123139:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775766490.123144:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775766490.123147:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766490.123152:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3867
00000100:00100000:2.0:1775766490.123156:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766490/real 1775766490]  req@ffff9f1485744e00 x1862022114513792/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766515 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766490.123156:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766490.123164:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766490.123172:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766490.123189:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744e00 x1862022114513792/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766515 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766490.123190:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747480 x1862022114514432/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766490.123218:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766490.123227:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775766490.123230:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766490.123238:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3867
00000100:00080000:2.0:1775766490.123240:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766490.123242:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766490.123248:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00080000:2.0:1775766490.123253:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000008:00100000:3.0:1775766490.123254:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775766490.123262:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114513792:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766490.123267:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744a80 x1862022114514560/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775766490.123277:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114513920:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775766490.123309:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766490.123322:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766490/real 1775766490]  req@ffff9f1485744380 x1862022114513920/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766515 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766490.123333:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744380 x1862022114513920/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766515 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766490.123340:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766490.123343:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766490.123346:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114513920:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766490.123350:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114514048:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775766490.123363:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766490.123369:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766490/real 1775766490]  req@ffff9f1485746a00 x1862022114514048/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766515 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766490.123390:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746a00 x1862022114514048/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766515 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766490.123406:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766490.123408:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766490.123411:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114514048:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775766490.123415:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114514176:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775766490.123429:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766490.123436:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766490/real 1775766490]  req@ffff9f1485746300 x1862022114514176/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766515 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766490.123462:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746300 x1862022114514176/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766515 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766490.123470:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766490.123473:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766490.123476:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114514176:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775766490.123494:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114514304:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775766490.123511:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766490.123520:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766490/real 1775766490]  req@ffff9f1485745880 x1862022114514304/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766515 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766490.123531:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745880 x1862022114514304/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766515 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766490.123539:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766490.123542:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766490.123545:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114514304:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766490.123551:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114514432:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775766490.123565:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766490.123617:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766490/real 1775766490]  req@ffff9f1485747480 x1862022114514432/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766515 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766490.123629:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747480 x1862022114514432/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766515 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766490.123637:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766490.123639:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766490.123643:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114514432:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766490.123648:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114514560:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775766490.123682:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766490.123689:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766490/real 1775766490]  req@ffff9f1485744a80 x1862022114514560/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766515 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766490.123698:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744a80 x1862022114514560/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766515 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766490.123704:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766490.123706:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766490.123709:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114514560:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775766490.378906:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766491.146600:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766491.402690:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766492.170650:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766492.426824:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766493.194653:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766493.450814:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766494.218655:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766494.474692:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766495.242627:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775766495.242649:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766495.242657:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775766495.242661:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766495.242671:0:8:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3872
00000100:00080000:3.0:1775766495.242682:0:8:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766495.242689:0:8:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775766495.242726:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746680 x1862022114514688/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766495.242763:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766495.242768:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775766495.242771:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766495.242777:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3872
00000100:00080000:3.0:1775766495.242782:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766495.242787:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775766495.242793:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114514688:192.168.202.110@tcp:250:kworker.0
00000008:00100000:3.0:1775766495.242798:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766495.242814:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744700 x1862022114514816/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766495.242826:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766495.242829:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775766495.242831:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766495.242835:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3872
00000100:00080000:3.0:1775766495.242848:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766495.242852:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766495.242858:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000800:00000100:2.0:1775766495.242865:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766495.242873:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745500 x1862022114514944/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766495.242884:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766495.242888:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775766495.242889:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766495.242894:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3872
00000100:00080000:3.0:1775766495.242898:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00100000:2.0:1775766495.242904:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766495/real 1775766495]  req@ffff9f1485746680 x1862022114514688/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766520 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766495.242904:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766495.242914:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775766495.242927:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746680 x1862022114514688/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766520 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766495.242931:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744000 x1862022114515072/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766495.242943:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766495.242951:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:3.0:1775766495.242953:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:2.0:1775766495.242957:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114514688:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775766495.242963:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775766495.242966:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766495.242972:0:8:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3872
00000100:00080000:3.0:1775766495.242977:0:8:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00100000:2.0:1775766495.242980:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114514816:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775766495.242986:0:8:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766495.242993:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:2.0:1775766495.243004:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766495.243011:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747800 x1862022114515200/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775766495.243016:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766495/real 1775766495]  req@ffff9f1485744700 x1862022114514816/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766520 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766495.243041:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744700 x1862022114514816/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766520 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766495.243043:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766495.243053:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775766495.243055:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766495.243058:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766495.243060:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766495.243069:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114514816:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775766495.243069:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3872
00000100:00100000:2.0:1775766495.243079:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114514944:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775766495.243079:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766495.243089:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766495.243097:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000800:00000100:2.0:1775766495.243100:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766495.243107:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766495/real 1775766495]  req@ffff9f1485745500 x1862022114514944/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766520 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766495.243118:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745500 x1862022114514944/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766520 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766495.243125:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00100000:3.0:1775766495.243129:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745c00 x1862022114515328/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766495.243130:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766495.243135:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114514944:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766495.243164:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114515072:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775766495.243182:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775766495.243183:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766495.243188:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775766495.243191:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:2.0:1775766495.243193:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766495/real 1775766495]  req@ffff9f1485744000 x1862022114515072/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766520 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766495.243201:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3872
00000100:00080000:3.0:1775766495.243209:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:2.0:1775766495.243212:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744000 x1862022114515072/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766520 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766495.243218:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766495.243228:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00080000:2.0:1775766495.243231:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766495.243238:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766495.243247:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745180 x1862022114515456/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775766495.243249:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114515072:192.168.202.110@tcp:8:kworker.0
00000100:00080000:3.0:1775766495.243266:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766495.243269:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00100000:2.0:1775766495.243273:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114515200:192.168.202.110@tcp:8:kworker.0
00000100:00080000:3.0:1775766495.243274:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766495.243284:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3872
00000100:00080000:3.0:1775766495.243289:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766495.243294:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000800:00000100:2.0:1775766495.243299:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000008:00100000:3.0:1775766495.243300:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775766495.243311:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766495/real 1775766495]  req@ffff9f1485747800 x1862022114515200/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766520 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766495.243317:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745f80 x1862022114515584/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775766495.243337:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747800 x1862022114515200/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766520 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766495.243347:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766495.243351:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766495.243356:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114515200:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775766495.243365:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114515328:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775766495.243386:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766495.243395:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766495/real 1775766495]  req@ffff9f1485745c00 x1862022114515328/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766520 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766495.243404:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745c00 x1862022114515328/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766520 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766495.243409:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766495.243411:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766495.243413:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114515328:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766495.243417:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114515456:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775766495.243428:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766495.243433:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766495/real 1775766495]  req@ffff9f1485745180 x1862022114515456/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766520 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766495.243440:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745180 x1862022114515456/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766520 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766495.243446:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766495.243448:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766495.243450:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114515456:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766495.243459:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114515584:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775766495.243469:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766495.243475:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766495/real 1775766495]  req@ffff9f1485745f80 x1862022114515584/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766520 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766495.243482:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745f80 x1862022114515584/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766520 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766495.243487:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766495.243489:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766495.243491:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114515584:192.168.202.110@tcp:8:kworker.0
00010000:00010000:3.0:1775766495.498740:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766496.266625:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775766496.522749:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766497.290628:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766497.546685:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766498.314619:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766498.570744:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766499.338684:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766499.594724:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766500.362679:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775766500.362684:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766500.362707:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775766500.362711:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766500.362719:0:8:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3877
00000100:00080000:3.0:1775766500.362730:0:8:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766500.362736:0:8:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775766500.362771:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747100 x1862022114515712/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766500.362796:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766500.362799:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775766500.362801:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766500.362815:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3877
00000100:00080000:3.0:1775766500.362818:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766500.362820:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766500.362829:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766500.362839:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745f80 x1862022114515840/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766500.362848:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:2.0:1775766500.362849:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114515712:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775766500.362852:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775766500.362855:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766500.362862:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3877
00000100:00080000:3.0:1775766500.362865:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766500.362867:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766500.362870:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766500.362879:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745180 x1862022114515968/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766500.362884:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766500.362886:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775766500.362887:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766500.362891:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3877
00000100:00080000:3.0:1775766500.362893:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766500.362896:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766500.362899:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766500.362908:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745c00 x1862022114516096/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766500.362930:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766500.362932:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775766500.362933:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766500.362935:0:8:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3877
00000100:00080000:3.0:1775766500.362938:0:8:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766500.362941:0:8:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766500.362944:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766500.362950:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747800 x1862022114516224/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:2.0:1775766500.362956:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775766500.362961:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766500.362964:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775766500.362965:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766500.362968:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3877
00000100:00080000:3.0:1775766500.362970:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766500.362973:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766500.362976:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766500.362982:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744000 x1862022114516352/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766500.362987:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766500.362990:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775766500.362991:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766500.362993:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3877
00000100:00080000:3.0:1775766500.362996:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766500.362998:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766500.363002:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766500.363008:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766500/real 1775766500]  req@ffff9f1485747100 x1862022114515712/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766525 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766500.363011:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745500 x1862022114516480/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766500.363035:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766500.363040:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775766500.363041:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766500.363044:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3877
00000100:00080000:3.0:1775766500.363047:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:2.0:1775766500.363051:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747100 x1862022114515712/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766525 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766500.363051:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766500.363057:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766500.363066:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744700 x1862022114516608/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766500.363068:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766500.363079:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766500.363110:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114515712:192.168.202.110@tcp:250:kworker.0
00000100:00100000:2.0:1775766500.363135:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114515840:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775766500.363179:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766500.363205:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766500/real 1775766500]  req@ffff9f1485745f80 x1862022114515840/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766525 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766500.363215:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745f80 x1862022114515840/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766525 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766500.363222:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766500.363224:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766500.363226:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114515840:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766500.363230:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114515968:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775766500.363244:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766500.363251:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766500/real 1775766500]  req@ffff9f1485745180 x1862022114515968/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766525 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766500.363259:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745180 x1862022114515968/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766525 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766500.363269:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766500.363271:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766500.363274:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114515968:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766500.363277:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114516096:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775766500.363288:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766500.363294:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766500/real 1775766500]  req@ffff9f1485745c00 x1862022114516096/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766525 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766500.363301:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745c00 x1862022114516096/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766525 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766500.363306:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766500.363308:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766500.363310:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114516096:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775766500.363314:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114516224:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775766500.363340:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766500.363345:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766500/real 1775766500]  req@ffff9f1485747800 x1862022114516224/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766525 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766500.363371:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747800 x1862022114516224/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766525 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766500.363376:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766500.363378:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766500.363381:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114516224:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775766500.363384:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114516352:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775766500.363396:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766500.363402:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766500/real 1775766500]  req@ffff9f1485744000 x1862022114516352/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766525 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766500.363410:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744000 x1862022114516352/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766525 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766500.363414:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766500.363416:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766500.363418:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114516352:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766500.363421:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114516480:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775766500.363431:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766500.363436:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766500/real 1775766500]  req@ffff9f1485745500 x1862022114516480/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766525 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766500.363443:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745500 x1862022114516480/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766525 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766500.363448:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766500.363450:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766500.363451:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114516480:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766500.363454:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114516608:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775766500.363463:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766500.363469:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766500/real 1775766500]  req@ffff9f1485744700 x1862022114516608/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766525 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766500.363476:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744700 x1862022114516608/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766525 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766500.363481:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766500.363482:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766500.363484:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114516608:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775766500.618677:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766501.386747:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766501.642791:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766502.410662:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775766502.666734:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766503.434788:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775766503.690734:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766504.458644:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766504.714709:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1775766505.482623:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766505.482632:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775766505.482636:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775766505.482645:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775766505.482645:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3882
00000100:00080000:3.0:1775766505.482655:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766505.482663:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775766505.482699:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746680 x1862022114516736/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766505.482722:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766505.482725:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775766505.482727:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766505.482730:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3882
00000100:00080000:3.0:1775766505.482732:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766505.482735:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766505.482742:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766505.482750:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744a80 x1862022114516864/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766505.482762:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766505.482765:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775766505.482766:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766505.482769:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3882
00000100:00080000:3.0:1775766505.482771:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766505.482774:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766505.482777:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766505.482784:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747480 x1862022114516992/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766505.482792:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:2.0:1775766505.482797:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114516736:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775766505.482797:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775766505.482798:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766505.482807:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3882
00000100:00080000:3.0:1775766505.482809:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766505.482812:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766505.482815:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766505.482826:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745880 x1862022114517120/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766505.482833:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766505.482836:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775766505.482837:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766505.482840:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3882
00000100:00080000:3.0:1775766505.482851:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766505.482855:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766505.482859:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:2.0:1775766505.482867:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766505.482871:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746300 x1862022114517248/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766505.482879:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766505.482882:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775766505.482884:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766505.482888:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3882
00000100:00080000:3.0:1775766505.482890:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766505.482893:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766505.482904:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766505.482918:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766505/real 1775766505]  req@ffff9f1485746680 x1862022114516736/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766530 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766505.482925:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746a00 x1862022114517376/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766505.482948:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766505.482951:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775766505.482952:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766505.482956:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3882
00000100:00080000:3.0:1775766505.482958:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:2.0:1775766505.482961:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746680 x1862022114516736/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766530 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766505.482972:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766505.482979:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766505.482992:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744380 x1862022114517504/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766505.482993:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766505.483004:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766505.483014:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114516736:192.168.202.110@tcp:250:kworker.0
00000100:00100000:2.0:1775766505.483028:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114516864:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775766505.483033:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766505.483052:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775766505.483055:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766505.483062:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3882
00000100:00080000:3.0:1775766505.483092:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000800:00000100:2.0:1775766505.483097:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775766505.483099:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766505.483105:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775766505.483113:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766505/real 1775766505]  req@ffff9f1485744a80 x1862022114516864/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766530 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766505.483119:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744e00 x1862022114517632/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775766505.483133:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744a80 x1862022114516864/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766530 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766505.483167:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766505.483170:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766505.483173:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114516864:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766505.483196:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114516992:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775766505.483212:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766505.483219:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766505/real 1775766505]  req@ffff9f1485747480 x1862022114516992/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766530 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766505.483228:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747480 x1862022114516992/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766530 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766505.483250:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766505.483270:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766505.483273:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114516992:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766505.483277:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114517120:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775766505.483292:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766505.483301:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766505/real 1775766505]  req@ffff9f1485745880 x1862022114517120/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766530 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766505.483309:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745880 x1862022114517120/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766530 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766505.483319:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766505.483321:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766505.483323:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114517120:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775766505.483327:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114517248:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775766505.483350:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766505.483356:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766505/real 1775766505]  req@ffff9f1485746300 x1862022114517248/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766530 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766505.483364:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746300 x1862022114517248/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766530 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766505.483369:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766505.483370:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766505.483372:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114517248:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775766505.483375:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114517376:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775766505.483385:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766505.483390:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766505/real 1775766505]  req@ffff9f1485746a00 x1862022114517376/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766530 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766505.483398:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746a00 x1862022114517376/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766530 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766505.483402:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766505.483404:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766505.483406:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114517376:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766505.483410:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114517504:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775766505.483419:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766505.483424:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766505/real 1775766505]  req@ffff9f1485744380 x1862022114517504/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766530 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766505.483431:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744380 x1862022114517504/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766530 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766505.483436:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766505.483438:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766505.483439:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114517504:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766505.483442:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114517632:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775766505.483451:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766505.483456:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766505/real 1775766505]  req@ffff9f1485744e00 x1862022114517632/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766530 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766505.483462:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744e00 x1862022114517632/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766530 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766505.483467:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766505.483469:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766505.483471:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114517632:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775766505.738795:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766506.506646:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766506.762690:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766507.530676:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766507.786785:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766508.554764:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766508.810673:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766509.578658:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775766509.834698:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1775766510.602649:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:2.0:1775766510.602654:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775766510.602663:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775766510.602667:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766510.602674:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3888
00000100:00080000:3.0:1775766510.602684:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766510.602690:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775766510.602724:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747b80 x1862022114517760/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766510.602745:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766510.602748:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775766510.602750:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766510.602753:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3888
00000100:00080000:3.0:1775766510.602755:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766510.602759:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766510.602767:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766510.602775:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114517760:192.168.202.110@tcp:250:kworker.0
00000100:00100000:3.0:1775766510.602781:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744e00 x1862022114517888/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766510.602789:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766510.602791:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775766510.602792:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766510.602795:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3888
00000100:00080000:3.0:1775766510.602797:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766510.602799:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766510.602801:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766510.602811:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744380 x1862022114518016/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766510.602817:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766510.602819:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775766510.602820:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766510.602823:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3888
00000100:00080000:3.0:1775766510.602826:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766510.602828:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766510.602832:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:2.0:1775766510.602836:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766510.602842:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746a00 x1862022114518144/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766510.602849:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766510.602852:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775766510.602854:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766510.602857:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3888
00000100:00080000:3.0:1775766510.602859:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766510.602862:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766510.602865:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775766510.602877:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766510/real 1775766510]  req@ffff9f1485747b80 x1862022114517760/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766535 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766510.602878:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746300 x1862022114518272/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766510.602918:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766510.602921:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775766510.602923:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:2.0:1775766510.602927:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747b80 x1862022114517760/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766535 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766510.602932:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3888
00000100:00080000:3.0:1775766510.602938:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766510.602946:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00080000:2.0:1775766510.602948:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000008:00100000:3.0:1775766510.602952:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00080000:2.0:1775766510.602956:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766510.602962:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114517760:192.168.202.110@tcp:250:kworker.0
00000100:00100000:3.0:1775766510.602971:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745880 x1862022114518400/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766510.602979:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766510.602982:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775766510.603000:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766510.603004:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3888
00000100:00080000:3.0:1775766510.603008:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:2.0:1775766510.603011:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114517888:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775766510.603015:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766510.603021:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766510.603034:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747480 x1862022114518528/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:2.0:1775766510.603041:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775766510.603049:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766510.603052:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775766510.603053:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:2.0:1775766510.603057:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766510/real 1775766510]  req@ffff9f1485744e00 x1862022114517888/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766535 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766510.603059:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3888
00000100:00080000:3.0:1775766510.603064:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766510.603070:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766510.603076:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775766510.603079:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744e00 x1862022114517888/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766535 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766510.603087:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744a80 x1862022114518656/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766510.603088:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766510.603096:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766510.603103:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114517888:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766510.603110:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114518016:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775766510.603124:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766510.603130:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766510/real 1775766510]  req@ffff9f1485744380 x1862022114518016/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766535 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766510.603137:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744380 x1862022114518016/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766535 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766510.603142:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766510.603144:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766510.603146:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114518016:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766510.603150:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114518144:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775766510.603160:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766510.603166:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766510/real 1775766510]  req@ffff9f1485746a00 x1862022114518144/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766535 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766510.603173:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746a00 x1862022114518144/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766535 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766510.603178:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766510.603180:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766510.603181:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114518144:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775766510.603185:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114518272:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775766510.603194:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766510.603199:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766510/real 1775766510]  req@ffff9f1485746300 x1862022114518272/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766535 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766510.603207:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746300 x1862022114518272/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766535 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766510.603212:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766510.603213:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766510.603215:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114518272:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775766510.603218:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114518400:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775766510.603227:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766510.603241:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766510/real 1775766510]  req@ffff9f1485745880 x1862022114518400/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766535 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766510.603248:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745880 x1862022114518400/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766535 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766510.603254:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766510.603256:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766510.603257:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114518400:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766510.603270:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114518528:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775766510.603281:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766510.603287:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766510/real 1775766510]  req@ffff9f1485747480 x1862022114518528/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766535 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766510.603294:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747480 x1862022114518528/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766535 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766510.603304:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766510.603305:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766510.603308:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114518528:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766510.603311:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114518656:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775766510.603320:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766510.603325:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766510/real 1775766510]  req@ffff9f1485744a80 x1862022114518656/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766535 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766510.603332:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744a80 x1862022114518656/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766535 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766510.603337:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766510.603338:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766510.603340:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114518656:192.168.202.110@tcp:8:kworker.0
00010000:00010000:3.0:1775766510.858731:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766511.626688:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766511.882732:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766512.650659:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766512.906724:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766513.674693:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766513.930758:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766514.698617:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766514.954708:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766515.722654:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:1.0:1775766515.722693:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766515.722700:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1775766515.722704:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766515.722715:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3893
00000100:00080000:1.0:1775766515.722727:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766515.722736:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775766515.722788:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff100 x1862022114518784/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766515.722821:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766515.722824:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775766515.722826:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766515.722830:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3893
00000100:00080000:1.0:1775766515.722834:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766515.722838:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766515.722849:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775766515.722865:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fd880 x1862022114518912/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775766515.722865:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114518784:192.168.202.110@tcp:250:kworker.0
00000100:00080000:1.0:1775766515.722882:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766515.722893:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775766515.722895:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766515.722900:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3893
00000100:00080000:1.0:1775766515.722904:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766515.722908:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766515.722913:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775766515.722925:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fe300 x1862022114519040/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766515.722945:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766515.722948:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775766515.722949:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766515.722953:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3893
00000100:00080000:1.0:1775766515.722957:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766515.722961:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766515.722965:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775766515.722976:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fc380 x1862022114519168/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:3.0:1775766515.722977:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:1.0:1775766515.722985:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766515.722988:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1775766515.722990:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766515.722992:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3893
00000100:00080000:1.0:1775766515.722995:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766515.722997:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766515.723000:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775766515.723006:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdc00 x1862022114519296/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766515.723012:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766515.723014:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775766515.723015:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766515.723019:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3893
00000100:00080000:1.0:1775766515.723022:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766515.723026:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775766515.723031:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766515/real 1775766515]  req@ffff9f14ab4ff100 x1862022114518784/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766540 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:1.0:1775766515.723032:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775766515.723043:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fed80 x1862022114519424/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766515.723049:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766515.723051:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775766515.723052:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766515.723054:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3893
00000100:00080000:1.0:1775766515.723056:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766515.723058:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766515.723060:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775766515.723065:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fca80 x1862022114519552/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766515.723070:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766515.723072:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775766515.723073:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766515.723075:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3893
00000100:00080000:1.0:1775766515.723077:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766515.723079:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766515.723081:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775766515.723086:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fce00 x1862022114519680/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775766515.723131:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff100 x1862022114518784/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766540 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766515.723144:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766515.723149:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766515.723154:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114518784:192.168.202.110@tcp:250:kworker.0
00000100:00100000:3.0:1775766515.723302:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114518912:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775766515.723328:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766515.723400:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766515/real 1775766515]  req@ffff9f14ab4fd880 x1862022114518912/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766540 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766515.723411:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fd880 x1862022114518912/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766540 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766515.723417:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766515.723420:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766515.723422:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114518912:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766515.723426:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114519040:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775766515.723443:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766515.723450:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766515/real 1775766515]  req@ffff9f14ab4fe300 x1862022114519040/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766540 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766515.723457:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fe300 x1862022114519040/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766540 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766515.723463:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766515.723465:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766515.723467:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114519040:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766515.723471:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114519168:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775766515.723482:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766515.723487:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766515/real 1775766515]  req@ffff9f14ab4fc380 x1862022114519168/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766540 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766515.723500:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fc380 x1862022114519168/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766540 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766515.723505:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766515.723507:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766515.723509:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114519168:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775766515.723512:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114519296:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775766515.723523:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766515.723528:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766515/real 1775766515]  req@ffff9f14ab4fdc00 x1862022114519296/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766540 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766515.723535:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdc00 x1862022114519296/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766540 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766515.723541:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766515.723542:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766515.723544:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114519296:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775766515.723547:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114519424:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775766515.723582:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766515.723592:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766515/real 1775766515]  req@ffff9f14ab4fed80 x1862022114519424/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766540 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766515.723601:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fed80 x1862022114519424/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766540 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766515.723611:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766515.723613:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766515.723615:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114519424:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766515.723619:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114519552:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775766515.723631:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766515.723637:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766515/real 1775766515]  req@ffff9f14ab4fca80 x1862022114519552/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766540 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766515.723646:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fca80 x1862022114519552/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766540 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766515.723652:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766515.723656:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766515.723660:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114519552:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766515.723666:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114519680:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775766515.723684:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766515.723695:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766515/real 1775766515]  req@ffff9f14ab4fce00 x1862022114519680/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766540 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766515.723708:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fce00 x1862022114519680/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766540 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766515.723716:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766515.723719:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766515.723723:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114519680:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775766515.978815:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766516.746670:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775766517.002739:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766517.770645:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775766518.026719:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766518.794639:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766519.050697:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766519.818635:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766520.074712:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766520.842689:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775766520.842697:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766520.842707:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775766520.842712:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766520.842723:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3898
00000100:00080000:3.0:1775766520.842735:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766520.842752:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775766520.842801:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746680 x1862022114519808/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766520.842830:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766520.842833:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775766520.842834:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766520.842838:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3898
00000100:00080000:3.0:1775766520.842840:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766520.842843:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766520.842853:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766520.842867:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744700 x1862022114519936/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775766520.842873:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114519808:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775766520.842891:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766520.842895:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775766520.842896:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766520.842900:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3898
00000100:00080000:3.0:1775766520.842904:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766520.842911:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766520.842915:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766520.842928:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745500 x1862022114520064/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766520.842935:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766520.842937:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775766520.842938:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766520.842941:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3898
00000100:00080000:3.0:1775766520.842945:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766520.842948:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766520.842952:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:2.0:1775766520.842956:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766520.842962:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744000 x1862022114520192/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766520.842970:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766520.842973:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775766520.842975:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766520.842979:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3898
00000100:00080000:3.0:1775766520.842986:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766520.842991:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766520.842996:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766520.843003:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747800 x1862022114520320/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775766520.843011:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766520/real 1775766520]  req@ffff9f1485746680 x1862022114519808/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766545 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766520.843051:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766520.843058:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775766520.843060:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766520.843065:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3898
00000100:00080000:3.0:1775766520.843069:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:2.0:1775766520.843072:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746680 x1862022114519808/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766545 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766520.843080:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766520.843093:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00080000:2.0:1775766520.843100:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00100000:3.0:1775766520.843116:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745c00 x1862022114520448/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766520.843117:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766520.843133:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114519808:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775766520.843148:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766520.843151:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775766520.843153:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766520.843158:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3898
00000100:00080000:3.0:1775766520.843161:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766520.843163:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766520.843168:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766520.843173:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114519936:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766520.843184:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745180 x1862022114520576/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766520.843193:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766520.843196:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775766520.843203:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000800:00000100:2.0:1775766520.843204:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775766520.843212:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3898
00000100:00080000:3.0:1775766520.843218:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766520.843225:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775766520.843228:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766520/real 1775766520]  req@ffff9f1485744700 x1862022114519936/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766545 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:3.0:1775766520.843229:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766520.843242:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745f80 x1862022114520704/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775766520.843267:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744700 x1862022114519936/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766545 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766520.843278:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766520.843283:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766520.843287:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114519936:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766520.843297:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114520064:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775766520.843320:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766520.843331:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766520/real 1775766520]  req@ffff9f1485745500 x1862022114520064/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766545 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766520.843342:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745500 x1862022114520064/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766545 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766520.843349:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766520.843352:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766520.843355:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114520064:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766520.843361:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114520192:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775766520.843378:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766520.843386:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766520/real 1775766520]  req@ffff9f1485744000 x1862022114520192/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766545 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766520.843397:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744000 x1862022114520192/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766545 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766520.843405:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766520.843408:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766520.843411:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114520192:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775766520.843418:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114520320:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775766520.843438:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766520.843447:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766520/real 1775766520]  req@ffff9f1485747800 x1862022114520320/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766545 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766520.843455:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747800 x1862022114520320/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766545 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766520.843461:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766520.843463:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766520.843465:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114520320:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775766520.843469:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114520448:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775766520.843480:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766520.843486:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766520/real 1775766520]  req@ffff9f1485745c00 x1862022114520448/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766545 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766520.843493:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745c00 x1862022114520448/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766545 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766520.843498:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766520.843500:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766520.843502:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114520448:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766520.843524:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114520576:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775766520.843535:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766520.843542:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766520/real 1775766520]  req@ffff9f1485745180 x1862022114520576/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766545 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766520.843549:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745180 x1862022114520576/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766545 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766520.843577:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766520.843579:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766520.843581:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114520576:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766520.843586:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114520704:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775766520.843598:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766520.843605:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766520/real 1775766520]  req@ffff9f1485745f80 x1862022114520704/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766545 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766520.843612:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745f80 x1862022114520704/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766545 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766520.843617:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766520.843619:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766520.843621:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114520704:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775766521.098783:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766521.866693:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766522.122771:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766522.890662:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766523.146772:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766523.914675:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775766524.170773:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766524.938700:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775766525.194762:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1775766525.962678:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:2.0:1775766525.962685:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775766525.962692:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775766525.962698:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766525.962711:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3903
00000100:00080000:3.0:1775766525.962726:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766525.962735:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775766525.962789:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747100 x1862022114520832/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766525.962867:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766525.962872:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775766525.962874:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766525.962880:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3903
00000100:00080000:3.0:1775766525.962884:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766525.962890:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766525.962903:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766525.962915:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114520832:192.168.202.110@tcp:250:kworker.0
00000100:00100000:3.0:1775766525.962924:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745f80 x1862022114520960/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766525.962946:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766525.962950:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775766525.962952:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766525.962958:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3903
00000100:00080000:3.0:1775766525.962972:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766525.962977:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766525.962984:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000800:00000100:2.0:1775766525.962989:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766525.963000:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745180 x1862022114521088/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766525.963012:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766525.963016:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775766525.963019:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766525.963023:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3903
00000100:00080000:3.0:1775766525.963028:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766525.963033:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775766525.963038:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766525/real 1775766525]  req@ffff9f1485747100 x1862022114520832/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766550 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:3.0:1775766525.963040:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766525.963060:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745c00 x1862022114521216/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775766525.963061:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747100 x1862022114520832/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766550 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766525.963094:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766525.963095:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766525.963099:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:3.0:1775766525.963103:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00100000:2.0:1775766525.963106:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114520832:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775766525.963108:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766525.963117:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3903
00000100:00080000:3.0:1775766525.963122:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766525.963127:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775766525.963132:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114520960:192.168.202.110@tcp:38:kworker.0
00000008:00100000:3.0:1775766525.963133:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:2.0:1775766525.963154:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766525.963164:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747800 x1862022114521344/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775766525.963166:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766525/real 1775766525]  req@ffff9f1485745f80 x1862022114520960/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766550 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766525.963198:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745f80 x1862022114520960/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766550 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766525.963201:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766525.963211:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775766525.963213:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766525.963215:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766525.963218:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766525.963223:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114520960:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775766525.963226:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3903
00000100:00100000:2.0:1775766525.963230:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114521088:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775766525.963234:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766525.963241:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000800:00000100:2.0:1775766525.963246:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000008:00100000:3.0:1775766525.963248:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766525.963254:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766525/real 1775766525]  req@ffff9f1485745180 x1862022114521088/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766550 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766525.963266:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745180 x1862022114521088/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766550 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766525.963266:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744000 x1862022114521472/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766525.963304:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766525.963305:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766525.963311:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00080000:3.0:1775766525.963314:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00100000:2.0:1775766525.963316:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114521088:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775766525.963318:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:2.0:1775766525.963323:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114521216:192.168.202.110@tcp:8:kworker.0
00000100:00080000:3.0:1775766525.963329:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3903
00000100:00080000:3.0:1775766525.963334:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766525.963340:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000800:00000100:2.0:1775766525.963342:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000008:00100000:3.0:1775766525.963350:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766525.963352:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766525/real 1775766525]  req@ffff9f1485745c00 x1862022114521216/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766550 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766525.963363:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745c00 x1862022114521216/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766550 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766525.963369:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745500 x1862022114521600/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766525.963371:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766525.963377:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766525.963393:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114521216:192.168.202.110@tcp:8:kworker.0
00000100:00080000:3.0:1775766525.963395:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766525.963400:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775766525.963403:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766525.963409:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3903
00000100:00100000:2.0:1775766525.963410:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114521344:192.168.202.110@tcp:8:kworker.0
00000100:00080000:3.0:1775766525.963418:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766525.963424:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000800:00000100:2.0:1775766525.963431:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000008:00100000:3.0:1775766525.963431:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775766525.963439:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766525/real 1775766525]  req@ffff9f1485747800 x1862022114521344/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766550 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766525.963448:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744700 x1862022114521728/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775766525.963457:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747800 x1862022114521344/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766550 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766525.963465:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766525.963469:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766525.963473:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114521344:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775766525.963478:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114521472:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775766525.963491:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766525.963497:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766525/real 1775766525]  req@ffff9f1485744000 x1862022114521472/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766550 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766525.963505:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744000 x1862022114521472/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766550 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766525.963510:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766525.963512:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766525.963514:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114521472:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766525.963517:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114521600:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775766525.963527:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766525.963532:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766525/real 1775766525]  req@ffff9f1485745500 x1862022114521600/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766550 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766525.963539:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745500 x1862022114521600/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766550 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766525.963544:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766525.963546:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766525.963547:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114521600:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766525.963575:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114521728:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775766525.963587:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766525.963594:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766525/real 1775766525]  req@ffff9f1485744700 x1862022114521728/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766550 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766525.963602:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744700 x1862022114521728/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766550 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766525.963608:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766525.963610:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766525.963612:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114521728:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775766526.218780:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766526.986656:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766527.242785:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766528.010659:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766528.266784:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766529.034642:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766529.290914:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766530.058667:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766530.314751:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766531.082642:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:1.0:1775766531.082657:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766531.082662:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1775766531.082666:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766531.082674:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3908
00000100:00080000:1.0:1775766531.082684:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766531.082690:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775766531.082721:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ffb80 x1862022114521856/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766531.082742:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766531.082744:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775766531.082746:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766531.082748:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3908
00000100:00080000:1.0:1775766531.082751:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766531.082753:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766531.082761:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766531.082783:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ffb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114521856:192.168.202.110@tcp:250:kworker.0
00000100:00100000:1.0:1775766531.082787:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdf80 x1862022114521984/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766531.082797:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766531.082799:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775766531.082800:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766531.082803:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3908
00000100:00080000:1.0:1775766531.082806:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766531.082808:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766531.082813:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775766531.082822:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff800 x1862022114522112/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766531.082829:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766531.082832:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775766531.082833:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766531.082835:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3908
00000100:00080000:1.0:1775766531.082837:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766531.082840:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766531.082843:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775766531.082853:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fea00 x1862022114522240/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766531.082862:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766531.082866:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1775766531.082868:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766531.082872:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3908
00000100:00080000:1.0:1775766531.082876:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00100000:2.0:1775766531.082877:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114521984:192.168.202.110@tcp:38:kworker.0
00000100:00080000:1.0:1775766531.082884:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766531.082890:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775766531.082903:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114522112:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775766531.082908:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fce00 x1862022114522368/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766531.082921:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:2.0:1775766531.082925:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114522240:192.168.202.110@tcp:8:kworker.0
00000100:00080000:1.0:1775766531.082927:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775766531.082931:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766531.082936:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3908
00000100:00080000:1.0:1775766531.082941:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:2.0:1775766531.082946:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114522368:192.168.202.110@tcp:8:kworker.0
00000100:00080000:1.0:1775766531.082947:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766531.082957:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775766531.082971:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fca80 x1862022114522496/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766531.082987:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766531.082991:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775766531.082993:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766531.082998:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3908
00000100:00080000:1.0:1775766531.083002:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:2.0:1775766531.083002:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114522496:192.168.202.110@tcp:38:kworker.0
00000100:00080000:1.0:1775766531.083011:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766531.083018:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775766531.083034:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fed80 x1862022114522624/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766531.083051:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766531.083055:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775766531.083057:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766531.083062:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3908
00000100:00100000:2.0:1775766531.083066:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114522624:192.168.202.110@tcp:38:kworker.0
00000100:00080000:1.0:1775766531.083069:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766531.083075:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766531.083080:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775766531.083092:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdc00 x1862022114522752/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775766531.083117:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114522752:192.168.202.110@tcp:8:kworker.0
00010000:00010000:3.0:1775766531.338750:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766532.106636:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775766532.362728:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766533.130637:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766533.386789:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766534.154628:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000400:00000100:2.0:1775766534.154917:0:2392:0:(lib-socket.c:454:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.202.110:988
00000400:00000100:2.0:1775766534.154973:0:2392:0:(acceptor.c:111:lnet_connect_console_error()) Connection to 192.168.202.110@tcp at host 192.168.202.110 was unreachable: the network or that node may be down, or Lustre may be misconfigured.
00000800:00000100:2.0:1775766534.154996:0:2392:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:2.0:1775766534.155050:0:2392:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:2.0:1775766534.155061:0:2392:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:2.0:1775766534.155074:0:2392:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:2.0:1775766534.155087:0:2392:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:2.0:1775766534.155099:0:2392:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000100:00100000:3.0:1775766534.155099:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766531/real 1775766534]  req@ffff9f14ab4fdf80 x1862022114521984/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766556 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000800:00000100:2.0:1775766534.155114:0:2392:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:2.0:1775766534.155138:0:2392:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000100:00100000:3.0:1775766534.155141:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdf80 x1862022114521984/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766556 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766534.155158:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766534.155165:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766534.155170:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114521984:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766534.155180:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766531/real 1775766534]  req@ffff9f14ab4ff800 x1862022114522112/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766556 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766534.155194:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff800 x1862022114522112/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766556 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766534.155210:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766534.155216:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766534.155219:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114522112:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766534.155226:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766531/real 1775766534]  req@ffff9f14ab4fea00 x1862022114522240/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766556 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766534.155240:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fea00 x1862022114522240/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766556 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766534.155247:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766534.155251:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766534.155255:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114522240:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775766534.155262:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766531/real 1775766534]  req@ffff9f14ab4ffb80 x1862022114521856/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766556 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766534.155275:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ffb80 x1862022114521856/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766556 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766534.155281:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766534.155285:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766534.155288:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ffb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114521856:192.168.202.110@tcp:250:kworker.0
00000100:00100000:3.0:1775766534.155326:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766531/real 1775766534]  req@ffff9f14ab4fdc00 x1862022114522752/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766556 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766534.155339:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdc00 x1862022114522752/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766556 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766534.155348:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766534.155352:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766534.155355:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114522752:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775766534.155362:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766531/real 1775766534]  req@ffff9f14ab4fed80 x1862022114522624/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766556 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766534.155375:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fed80 x1862022114522624/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766556 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766534.155383:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766534.155387:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766534.155391:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114522624:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766534.155400:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766531/real 1775766534]  req@ffff9f14ab4fca80 x1862022114522496/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766556 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766534.155413:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fca80 x1862022114522496/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766556 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766534.155422:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766534.155426:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766534.155430:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114522496:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766534.155437:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766531/real 1775766534]  req@ffff9f14ab4fce00 x1862022114522368/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766556 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766534.155449:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fce00 x1862022114522368/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766556 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766534.155458:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766534.155461:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766534.155464:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114522368:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775766534.410806:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766535.178684:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766535.434840:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766536.202618:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775766536.202642:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766536.202647:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775766536.202651:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766536.202658:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3913
00000100:00080000:3.0:1775766536.202668:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766536.202674:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775766536.202714:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746680 x1862022114522880/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766536.202754:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766536.202758:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775766536.202760:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766536.202765:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3913
00000100:00080000:3.0:1775766536.202769:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766536.202774:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766536.202786:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766536.202802:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744a80 x1862022114523008/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775766536.202810:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114522880:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775766536.202817:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766536.202822:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775766536.202824:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766536.202829:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3913
00000100:00080000:3.0:1775766536.202834:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766536.202837:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766536.202843:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766536.202856:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747480 x1862022114523136/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766536.202867:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766536.202870:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775766536.202886:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000800:00000100:2.0:1775766536.202894:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775766536.202895:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3913
00000100:00080000:3.0:1775766536.202903:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766536.202909:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766536.202915:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766536.202928:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745880 x1862022114523264/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766536.202940:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:2.0:1775766536.202945:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766536/real 1775766536]  req@ffff9f1485746680 x1862022114522880/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766561 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766536.202945:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775766536.202949:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766536.202958:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3913
00000100:00080000:3.0:1775766536.202965:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766536.202971:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766536.202977:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775766536.202979:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746680 x1862022114522880/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766561 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766536.202991:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746300 x1862022114523392/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766536.202992:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766536.203002:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:3.0:1775766536.203008:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:2.0:1775766536.203010:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114522880:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775766536.203016:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775766536.203019:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766536.203025:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3913
00000100:00080000:3.0:1775766536.203029:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766536.203034:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766536.203040:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766536.203043:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114523008:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766536.203055:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746a00 x1862022114523520/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766536.203065:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766536.203069:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775766536.203071:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766536.203076:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3913
00000800:00000100:2.0:1775766536.203080:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775766536.203089:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766536.203094:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766536.203100:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766536.203101:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766536/real 1775766536]  req@ffff9f1485744a80 x1862022114523008/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766561 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766536.203117:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744380 x1862022114523648/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775766536.203121:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744a80 x1862022114523008/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766561 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766536.203138:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766536.203141:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766536.203143:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775766536.203148:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00080000:3.0:1775766536.203148:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:2.0:1775766536.203157:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114523008:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775766536.203158:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3913
00000100:00080000:3.0:1775766536.203164:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:2.0:1775766536.203170:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114523136:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775766536.203171:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766536.203180:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766536.203195:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744e00 x1862022114523776/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:2.0:1775766536.203206:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766536.203223:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766536/real 1775766536]  req@ffff9f1485747480 x1862022114523136/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766561 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766536.203237:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747480 x1862022114523136/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766561 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766536.203246:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766536.203248:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766536.203251:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114523136:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766536.203259:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114523264:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775766536.203281:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766536.203292:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766536/real 1775766536]  req@ffff9f1485745880 x1862022114523264/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766561 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766536.203305:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745880 x1862022114523264/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766561 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766536.203314:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766536.203317:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766536.203320:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114523264:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775766536.203326:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114523392:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775766536.203357:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766536.203368:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766536/real 1775766536]  req@ffff9f1485746300 x1862022114523392/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766561 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766536.203382:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746300 x1862022114523392/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766561 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766536.203390:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766536.203393:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766536.203396:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114523392:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775766536.203417:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114523520:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775766536.203442:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766536.203455:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766536/real 1775766536]  req@ffff9f1485746a00 x1862022114523520/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766561 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766536.203467:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746a00 x1862022114523520/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766561 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766536.203483:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766536.203486:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766536.203490:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114523520:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766536.203496:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114523648:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775766536.203518:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766536.203530:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766536/real 1775766536]  req@ffff9f1485744380 x1862022114523648/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766561 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766536.203542:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744380 x1862022114523648/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766561 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766536.203550:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766536.203579:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766536.203584:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114523648:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766536.203591:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114523776:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775766536.203611:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766536.203623:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766536/real 1775766536]  req@ffff9f1485744e00 x1862022114523776/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766561 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766536.203635:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744e00 x1862022114523776/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766561 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766536.203652:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766536.203655:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766536.203659:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114523776:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775766536.458682:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766537.226666:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766537.482763:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766538.250672:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775766538.506753:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766539.274641:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775766539.530826:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766540.298668:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766540.554737:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1775766541.322636:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766541.322646:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775766541.322650:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766541.322658:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3918
00000100:00080000:3.0:1775766541.322681:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766541.322688:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775766541.322713:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:3.0:1775766541.322743:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747b80 x1862022114523904/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766541.322774:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766541.322778:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775766541.322780:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766541.322784:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3918
00000100:00080000:3.0:1775766541.322788:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766541.322792:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766541.322803:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766541.322818:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114523904:192.168.202.110@tcp:250:kworker.0
00000100:00100000:3.0:1775766541.322822:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744e00 x1862022114524032/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766541.322844:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766541.322848:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775766541.322850:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766541.322856:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3918
00000100:00080000:3.0:1775766541.322859:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766541.322861:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766541.322866:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766541.322879:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744380 x1862022114524160/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766541.322888:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766541.322892:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775766541.322895:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766541.322899:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3918
00000100:00080000:3.0:1775766541.322902:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766541.322905:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766541.322909:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:2.0:1775766541.322919:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766541.322930:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746a00 x1862022114524288/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766541.322945:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766541.322949:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775766541.322951:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766541.322955:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3918
00000100:00080000:3.0:1775766541.322960:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766541.322964:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766541.322969:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775766541.322978:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766541/real 1775766541]  req@ffff9f1485747b80 x1862022114523904/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766566 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766541.322984:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746300 x1862022114524416/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766541.323051:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766541.323057:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775766541.323059:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:2.0:1775766541.323061:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747b80 x1862022114523904/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766566 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766541.323072:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3918
00000100:00080000:3.0:1775766541.323082:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766541.323094:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00080000:2.0:1775766541.323095:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000008:00100000:3.0:1775766541.323099:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00080000:2.0:1775766541.323103:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766541.323121:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745880 x1862022114524544/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775766541.323123:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114523904:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775766541.323150:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766541.323156:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775766541.323159:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:2.0:1775766541.323162:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114524032:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775766541.323172:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3918
00000800:00000100:2.0:1775766541.323190:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775766541.323190:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766541.323204:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775766541.323210:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766541/real 1775766541]  req@ffff9f1485744e00 x1862022114524032/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766566 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:3.0:1775766541.323212:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766541.323241:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744e00 x1862022114524032/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766566 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766541.323248:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766541.323256:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766541.323257:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747480 x1862022114524672/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775766541.323267:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114524032:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766541.323291:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114524160:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775766541.323319:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766541.323323:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000800:00000100:2.0:1775766541.323325:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775766541.323330:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766541.323338:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3918
00000100:00080000:3.0:1775766541.323343:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:2.0:1775766541.323344:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766541/real 1775766541]  req@ffff9f1485744380 x1862022114524160/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766566 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766541.323357:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775766541.323361:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744380 x1862022114524160/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766566 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766541.323369:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000008:00100000:3.0:1775766541.323369:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00080000:2.0:1775766541.323376:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766541.323381:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114524160:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766541.323387:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114524288:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775766541.323398:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744a80 x1862022114524800/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:2.0:1775766541.323408:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766541.323417:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766541/real 1775766541]  req@ffff9f1485746a00 x1862022114524288/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766566 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766541.323437:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746a00 x1862022114524288/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766566 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766541.323447:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766541.323450:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766541.323451:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114524288:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775766541.323456:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114524416:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775766541.323469:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766541.323475:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766541/real 1775766541]  req@ffff9f1485746300 x1862022114524416/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766566 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766541.323481:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746300 x1862022114524416/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766566 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766541.323485:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766541.323488:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766541.323490:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114524416:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775766541.323494:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114524544:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775766541.323504:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766541.323510:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766541/real 1775766541]  req@ffff9f1485745880 x1862022114524544/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766566 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766541.323516:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745880 x1862022114524544/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766566 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766541.323522:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766541.323523:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766541.323526:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114524544:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766541.323543:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114524672:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775766541.323577:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766541.323585:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766541/real 1775766541]  req@ffff9f1485747480 x1862022114524672/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766566 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766541.323593:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747480 x1862022114524672/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766566 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766541.323598:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766541.323601:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766541.323603:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114524672:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766541.323607:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114524800:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775766541.323625:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766541.323635:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766541/real 1775766541]  req@ffff9f1485744a80 x1862022114524800/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766566 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766541.323648:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744a80 x1862022114524800/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766566 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766541.323656:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766541.323660:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766541.323678:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114524800:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775766541.578765:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766542.346830:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766542.602890:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766543.370628:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766543.626838:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766544.394727:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766544.650789:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766545.418640:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775766545.674785:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766546.442662:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775766546.442670:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766546.442677:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775766546.442681:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766546.442689:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3923
00000100:00080000:3.0:1775766546.442699:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766546.442705:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775766546.442739:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746680 x1862022114524928/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766546.442763:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766546.442766:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775766546.442767:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766546.442770:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3923
00000100:00080000:3.0:1775766546.442773:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766546.442777:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766546.442793:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766546.442797:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114524928:192.168.202.110@tcp:250:kworker.0
00000100:00100000:3.0:1775766546.442823:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744700 x1862022114525056/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766546.442833:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766546.442837:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775766546.442838:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766546.442841:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3923
00000100:00080000:3.0:1775766546.442845:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766546.442848:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766546.442853:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766546.442862:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745500 x1862022114525184/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766546.442870:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766546.442873:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775766546.442874:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766546.442878:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3923
00000800:00000100:2.0:1775766546.442884:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775766546.442884:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766546.442891:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766546.442895:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766546.442905:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744000 x1862022114525312/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766546.442912:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766546.442915:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775766546.442917:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766546.442921:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3923
00000100:00080000:3.0:1775766546.442925:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766546.442929:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775766546.442930:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766546/real 1775766546]  req@ffff9f1485746680 x1862022114524928/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766571 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:3.0:1775766546.442934:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766546.442944:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747800 x1862022114525440/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766546.442968:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766546.442972:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775766546.442974:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:2.0:1775766546.442976:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746680 x1862022114524928/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766571 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766546.442996:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3923
00000100:00080000:3.0:1775766546.443003:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766546.443011:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00080000:2.0:1775766546.443015:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000008:00100000:3.0:1775766546.443016:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00080000:2.0:1775766546.443026:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766546.443031:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745c00 x1862022114525568/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775766546.443038:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114524928:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775766546.443076:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766546.443081:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775766546.443084:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:2.0:1775766546.443086:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114525056:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775766546.443092:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3923
00000100:00080000:3.0:1775766546.443099:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766546.443104:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766546.443111:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000800:00000100:2.0:1775766546.443117:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766546.443130:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745180 x1862022114525696/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775766546.443131:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766546/real 1775766546]  req@ffff9f1485744700 x1862022114525056/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766571 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766546.443184:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744700 x1862022114525056/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766571 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766546.443186:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766546.443198:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775766546.443202:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766546.443202:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766546.443211:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00080000:3.0:1775766546.443215:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3923
00000100:00100000:2.0:1775766546.443219:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114525056:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775766546.443226:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:2.0:1775766546.443227:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114525184:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775766546.443237:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766546.443244:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:2.0:1775766546.443250:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766546.443260:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766546/real 1775766546]  req@ffff9f1485745500 x1862022114525184/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766571 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766546.443265:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745f80 x1862022114525824/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775766546.443301:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745500 x1862022114525184/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766571 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766546.443313:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766546.443318:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766546.443323:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114525184:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766546.443331:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114525312:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775766546.443356:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766546.443369:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766546/real 1775766546]  req@ffff9f1485744000 x1862022114525312/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766571 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766546.443381:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744000 x1862022114525312/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766571 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766546.443387:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766546.443389:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766546.443391:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114525312:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775766546.443396:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114525440:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775766546.443411:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766546.443418:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766546/real 1775766546]  req@ffff9f1485747800 x1862022114525440/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766571 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766546.443426:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747800 x1862022114525440/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766571 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766546.443432:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766546.443434:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766546.443436:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114525440:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775766546.443440:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114525568:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775766546.443450:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766546.443457:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766546/real 1775766546]  req@ffff9f1485745c00 x1862022114525568/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766571 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766546.443481:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745c00 x1862022114525568/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766571 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766546.443487:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766546.443489:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766546.443491:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114525568:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766546.443506:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114525696:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775766546.443520:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766546.443527:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766546/real 1775766546]  req@ffff9f1485745180 x1862022114525696/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766571 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766546.443536:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745180 x1862022114525696/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766571 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766546.443542:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766546.443545:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766546.443548:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114525696:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766546.443552:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114525824:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775766546.443562:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766546.443568:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766546/real 1775766546]  req@ffff9f1485745f80 x1862022114525824/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766571 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766546.443576:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745f80 x1862022114525824/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766571 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766546.443599:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766546.443602:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766546.443605:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114525824:192.168.202.110@tcp:8:kworker.0
00010000:00010000:3.0:1775766546.698877:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766547.466678:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766547.722887:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766548.490648:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766548.746800:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766549.514660:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766549.770736:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766550.538662:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766550.794787:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766551.562808:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775766551.562822:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766551.562831:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775766551.562838:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766551.562849:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3929
00000100:00080000:3.0:1775766551.562862:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766551.562873:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775766551.562912:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747100 x1862022114525952/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766551.562941:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766551.562945:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775766551.562947:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766551.562953:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3929
00000100:00080000:3.0:1775766551.562958:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766551.562965:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766551.562988:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766551.563021:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745f80 x1862022114526080/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766551.563036:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766551.563043:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775766551.563045:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:2.0:1775766551.563059:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114525952:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775766551.563062:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3929
00000100:00080000:3.0:1775766551.563074:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766551.563081:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766551.563087:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766551.563103:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745180 x1862022114526208/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766551.563113:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766551.563117:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775766551.563119:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766551.563124:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3929
00000100:00080000:3.0:1775766551.563132:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766551.563140:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766551.563149:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:2.0:1775766551.563186:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766551.563249:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766551/real 1775766551]  req@ffff9f1485747100 x1862022114525952/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766576 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766551.563259:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745c00 x1862022114526336/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775766551.563274:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747100 x1862022114525952/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766576 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766551.563299:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766551.563301:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766551.563311:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:3.0:1775766551.563312:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775766551.563318:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:2.0:1775766551.563319:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114525952:192.168.202.110@tcp:250:kworker.0
00000100:00100000:2.0:1775766551.563363:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114526080:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775766551.563388:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766551.563400:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766551/real 1775766551]  req@ffff9f1485745f80 x1862022114526080/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766576 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766551.563413:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745f80 x1862022114526080/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766576 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766551.563422:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766551.563425:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766551.563428:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114526080:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766551.563445:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114526208:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775766551.563464:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766551.563473:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766551/real 1775766551]  req@ffff9f1485745180 x1862022114526208/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766576 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766551.563486:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745180 x1862022114526208/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766576 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766551.563501:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766551.563511:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00080000:3.0:1775766551.563519:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3929
00000100:00100000:2.0:1775766551.563523:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114526208:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766551.563549:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114526336:192.168.202.110@tcp:8:kworker.0
00000100:00080000:3.0:1775766551.563551:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000800:00000100:2.0:1775766551.563663:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766551.563679:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766551/real 1775766551]  req@ffff9f1485745c00 x1862022114526336/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766576 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766551.563695:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745c00 x1862022114526336/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766576 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766551.563718:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766551.563731:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00080000:3.0:1775766551.563739:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775766551.563750:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114526336:192.168.202.110@tcp:8:kworker.0
00000008:00100000:3.0:1775766551.563755:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766551.563814:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747800 x1862022114526464/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766551.563838:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766551.563843:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775766551.563846:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766551.563851:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3929
00000100:00100000:2.0:1775766551.563858:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114526464:192.168.202.110@tcp:8:kworker.0
00000100:00080000:3.0:1775766551.563863:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766551.563871:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766551.563878:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766551.563893:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744000 x1862022114526592/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:2.0:1775766551.563901:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775766551.563939:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766551.563944:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775766551.563947:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766551.563954:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3929
00000100:00100000:2.0:1775766551.563961:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766551/real 1775766551]  req@ffff9f1485747800 x1862022114526464/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766576 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766551.563964:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766551.563984:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766551.564003:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766551.564011:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747800 x1862022114526464/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766576 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766551.564023:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745500 x1862022114526720/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766551.564065:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766551.564078:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775766551.564083:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766551.564089:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3929
00000100:00080000:2.0:1775766551.564091:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766551.564093:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766551.564101:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00080000:2.0:1775766551.564109:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000008:00100000:3.0:1775766551.564109:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775766551.564123:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114526464:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775766551.564126:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744700 x1862022114526848/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775766551.564154:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114526592:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775766551.564187:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766551.564201:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766551/real 1775766551]  req@ffff9f1485744000 x1862022114526592/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766576 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766551.564215:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744000 x1862022114526592/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766576 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766551.564226:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766551.564230:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766551.564234:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114526592:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766551.564240:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114526720:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775766551.564257:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766551.564267:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766551/real 1775766551]  req@ffff9f1485745500 x1862022114526720/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766576 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766551.564278:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745500 x1862022114526720/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766576 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766551.564285:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766551.564289:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766551.564293:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114526720:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766551.564307:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114526848:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775766551.564324:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766551.564335:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766551/real 1775766551]  req@ffff9f1485744700 x1862022114526848/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766576 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766551.564348:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744700 x1862022114526848/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766576 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766551.564357:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766551.564360:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766551.564364:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114526848:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775766551.818913:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766552.586661:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775766552.842746:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766553.610665:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775766553.866759:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766554.634673:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766554.890728:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766555.658655:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766555.914706:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766556.682669:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:0.0:1775766556.682690:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766556.682699:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1775766556.682703:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766556.682712:0:8:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3934
00000100:00080000:0.0:1775766556.682722:0:8:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766556.682727:0:8:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775766556.682762:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3480 x1862022114526976/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775766556.682791:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766556.682794:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775766556.682795:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766556.682798:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3934
00000100:00080000:0.0:1775766556.682801:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766556.682803:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775766556.682811:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775766556.682821:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0000 x1862022114527104/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775766556.682830:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766556.682834:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:0.0:1775766556.682836:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:2.0:1775766556.682842:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114526976:192.168.202.110@tcp:250:kworker.0
00000100:00080000:0.0:1775766556.682845:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3934
00000100:00080000:0.0:1775766556.682857:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766556.682860:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775766556.682863:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775766556.682873:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d1500 x1862022114527232/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775766556.682879:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766556.682882:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775766556.682883:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766556.682886:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3934
00000100:00080000:0.0:1775766556.682889:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766556.682892:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775766556.682896:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775766556.682902:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3800 x1862022114527360/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775766556.682908:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766556.682910:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1775766556.682911:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766556.682914:0:8:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3934
00000100:00080000:0.0:1775766556.682920:0:8:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766556.682922:0:8:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775766556.682926:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:2.0:1775766556.682934:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766556.682939:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d1180 x1862022114527488/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775766556.682946:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766556.682949:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775766556.682951:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766556.682956:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3934
00000100:00080000:0.0:1775766556.682962:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766556.682966:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775766556.682968:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775766556.682978:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d2300 x1862022114527616/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775766556.682987:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:2.0:1775766556.682989:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766556/real 1775766556]  req@ffff9f14a04d3480 x1862022114526976/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766581 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766556.682992:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:0.0:1775766556.682998:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766556.683017:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3934
00000100:00080000:0.0:1775766556.683025:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766556.683030:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775766556.683035:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766556.683040:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3480 x1862022114526976/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766581 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766556.683050:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0e00 x1862022114527744/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766556.683063:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766556.683078:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:0.0:1775766556.683080:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:2.0:1775766556.683085:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114526976:192.168.202.110@tcp:250:kworker.0
00000100:00080000:0.0:1775766556.683093:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775766556.683096:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766556.683103:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3934
00000100:00080000:0.0:1775766556.683110:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766556.683126:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775766556.683133:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775766556.683139:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114527104:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766556.683150:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0380 x1862022114527872/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:2.0:1775766556.683166:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766556.683181:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766556/real 1775766556]  req@ffff9f14a04d0000 x1862022114527104/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766581 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766556.683192:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0000 x1862022114527104/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766581 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766556.683198:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766556.683201:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766556.683204:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114527104:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766556.683208:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114527232:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775766556.683220:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766556.683225:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766556/real 1775766556]  req@ffff9f14a04d1500 x1862022114527232/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766581 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766556.683233:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1500 x1862022114527232/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766581 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766556.683238:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766556.683240:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766556.683242:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114527232:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766556.683245:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114527360:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775766556.683256:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766556.683262:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766556/real 1775766556]  req@ffff9f14a04d3800 x1862022114527360/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766581 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766556.683270:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3800 x1862022114527360/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766581 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766556.683275:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766556.683277:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766556.683279:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114527360:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775766556.683290:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114527488:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775766556.683301:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766556.683307:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766556/real 1775766556]  req@ffff9f14a04d1180 x1862022114527488/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766581 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766556.683314:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1180 x1862022114527488/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766581 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766556.683319:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766556.683322:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766556.683323:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114527488:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775766556.683327:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114527616:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775766556.683338:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766556.683343:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766556/real 1775766556]  req@ffff9f14a04d2300 x1862022114527616/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766581 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766556.683350:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d2300 x1862022114527616/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766581 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766556.683376:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766556.683378:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766556.683381:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114527616:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766556.683385:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114527744:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775766556.683397:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766556.683404:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766556/real 1775766556]  req@ffff9f14a04d0e00 x1862022114527744/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766581 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766556.683411:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0e00 x1862022114527744/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766581 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766556.683416:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766556.683418:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766556.683420:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114527744:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766556.683437:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114527872:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775766556.683448:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766556.683454:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766556/real 1775766556]  req@ffff9f14a04d0380 x1862022114527872/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766581 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766556.683462:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0380 x1862022114527872/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766581 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766556.683467:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766556.683468:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766556.683471:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114527872:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775766556.938754:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766557.706602:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766557.962710:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766558.730607:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766558.986720:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766559.754660:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775766560.010744:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766560.778641:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775766561.034799:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766561.802691:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:2.0:1775766561.802744:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766561.802750:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1775766561.802755:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766561.802765:0:8:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3939
00000100:00080000:2.0:1775766561.802789:0:8:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766561.802798:0:8:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775766561.802854:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2d80 x1862022114528000/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766561.802896:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766561.802900:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775766561.802902:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766561.802918:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3939
00000100:00080000:2.0:1775766561.802922:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766561.802926:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766561.802937:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766561.802950:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0380 x1862022114528128/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766561.802970:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766561.802985:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775766561.802987:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766561.802996:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3939
00000100:00100000:3.0:1775766561.802998:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114528000:192.168.202.110@tcp:250:kworker.0
00000100:00080000:2.0:1775766561.803000:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766561.803010:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766561.803016:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766561.803025:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1500 x1862022114528256/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766561.803035:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766561.803038:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775766561.803040:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766561.803044:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3939
00000100:00080000:2.0:1775766561.803049:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766561.803052:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766561.803058:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775766561.803070:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1f80 x1862022114528384/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766561.803081:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766561.803085:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1775766561.803087:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766561.803090:0:8:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3939
00000100:00080000:2.0:1775766561.803095:0:8:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766561.803102:0:8:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000800:00000100:3.0:1775766561.803103:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000008:00100000:2.0:1775766561.803111:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775766561.803141:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0a80 x1862022114528512/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766561.803157:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766561.803164:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775766561.803167:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:3.0:1775766561.803171:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766561/real 1775766561]  req@ffff9f14852d2d80 x1862022114528000/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766586 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766561.803179:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3939
00000100:00080000:2.0:1775766561.803192:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766561.803202:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775766561.803208:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2d80 x1862022114528000/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766586 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:2.0:1775766561.803213:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00080000:3.0:1775766561.803237:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766561.803242:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766561.803246:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114528000:192.168.202.110@tcp:250:kworker.0
00000100:00100000:2.0:1775766561.803258:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2680 x1862022114528640/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775766561.803260:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114528128:192.168.202.110@tcp:38:kworker.0
00000100:00080000:2.0:1775766561.803282:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766561.803286:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775766561.803289:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766561.803296:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3939
00000800:00000100:3.0:1775766561.803304:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:2.0:1775766561.803307:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766561.803315:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766561.803323:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766561.803325:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766561/real 1775766561]  req@ffff9f14852d0380 x1862022114528128/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766586 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766561.803341:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1180 x1862022114528768/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775766561.803347:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0380 x1862022114528128/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766586 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766561.803392:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766561.803410:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00080000:2.0:1775766561.803413:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:3.0:1775766561.803416:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114528128:192.168.202.110@tcp:38:kworker.0
00000100:00080000:2.0:1775766561.803426:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775766561.803432:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766561.803441:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3939
00000100:00080000:2.0:1775766561.803447:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766561.803487:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766561.803497:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766561.803504:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114528256:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766561.803525:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2a00 x1862022114528896/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:3.0:1775766561.803539:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766561.803589:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766561/real 1775766561]  req@ffff9f14852d1500 x1862022114528256/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766586 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766561.803608:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1500 x1862022114528256/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766586 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766561.803618:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766561.803623:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766561.803630:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114528256:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766561.803639:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114528384:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775766561.803661:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766561.803672:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766561/real 1775766561]  req@ffff9f14852d1f80 x1862022114528384/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766586 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766561.803685:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1f80 x1862022114528384/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766586 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766561.803693:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766561.803697:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766561.803700:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114528384:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775766561.803705:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114528512:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775766561.803721:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766561.803730:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766561/real 1775766561]  req@ffff9f14852d0a80 x1862022114528512/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766586 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766561.803741:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0a80 x1862022114528512/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766586 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766561.803749:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766561.803752:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766561.803755:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114528512:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775766561.803760:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114528640:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775766561.803776:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766561.803789:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766561/real 1775766561]  req@ffff9f14852d2680 x1862022114528640/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766586 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766561.803801:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2680 x1862022114528640/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766586 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766561.803810:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766561.803813:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766561.803816:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114528640:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766561.803821:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114528768:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775766561.803835:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766561.803843:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766561/real 1775766561]  req@ffff9f14852d1180 x1862022114528768/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766586 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766561.803855:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1180 x1862022114528768/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766586 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766561.803864:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766561.803867:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766561.803869:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114528768:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766561.803889:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114528896:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775766561.803914:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766561.803923:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766561/real 1775766561]  req@ffff9f14852d2a00 x1862022114528896/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766586 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766561.803935:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2a00 x1862022114528896/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766586 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766561.803950:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766561.803953:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766561.803956:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114528896:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775766562.058693:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766562.826675:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766563.082817:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766563.850739:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766564.106757:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766564.874679:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766565.130687:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766565.898712:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766566.154731:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766566.922693:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775766566.922710:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766566.922737:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775766566.922743:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766566.922818:0:8:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3944
00000100:00080000:3.0:1775766566.922832:0:8:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766566.922865:0:8:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775766566.922912:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746680 x1862022114529024/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766566.922959:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766566.922963:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775766566.922964:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766566.922968:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3944
00000100:00080000:3.0:1775766566.922971:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766566.922975:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766566.922985:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766566.922996:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744a80 x1862022114529152/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766566.923008:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:2.0:1775766566.923009:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114529024:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775766566.923014:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775766566.923018:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766566.923024:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3944
00000100:00080000:3.0:1775766566.923027:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766566.923030:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766566.923034:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766566.923045:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747480 x1862022114529280/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766566.923053:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766566.923056:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775766566.923058:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766566.923061:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3944
00000100:00080000:3.0:1775766566.923079:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766566.923083:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766566.923087:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766566.923097:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745880 x1862022114529408/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:2.0:1775766566.923106:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775766566.923111:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766566.923115:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775766566.923116:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766566.923121:0:8:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3944
00000100:00080000:3.0:1775766566.923124:0:8:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766566.923127:0:8:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766566.923131:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766566.923140:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746300 x1862022114529536/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766566.923148:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766566.923151:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775766566.923152:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766566.923156:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3944
00000100:00080000:3.0:1775766566.923160:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766566.923163:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766566.923167:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766566.923170:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766566/real 1775766566]  req@ffff9f1485746680 x1862022114529024/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766591 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766566.923178:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746a00 x1862022114529664/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766566.923200:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766566.923207:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775766566.923209:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766566.923214:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3944
00000100:00080000:3.0:1775766566.923218:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:2.0:1775766566.923221:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746680 x1862022114529024/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766591 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766566.923226:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766566.923234:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766566.923246:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744380 x1862022114529792/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766566.923252:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766566.923258:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766566.923262:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:3.0:1775766566.923263:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775766566.923268:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:2.0:1775766566.923271:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114529024:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775766566.923276:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3944
00000100:00080000:3.0:1775766566.923282:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766566.923284:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766566.923288:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766566.923295:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744e00 x1862022114529920/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775766566.923305:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114529152:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775766566.923359:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766566.923379:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766566/real 1775766566]  req@ffff9f1485744a80 x1862022114529152/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766591 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766566.923430:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744a80 x1862022114529152/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766591 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766566.923442:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766566.923447:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766566.923452:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114529152:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766566.923462:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114529280:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775766566.923509:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766566.923523:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766566/real 1775766566]  req@ffff9f1485747480 x1862022114529280/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766591 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766566.923534:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747480 x1862022114529280/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766591 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766566.923542:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766566.923545:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766566.923547:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114529280:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766566.923554:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114529408:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775766566.923572:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766566.923580:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766566/real 1775766566]  req@ffff9f1485745880 x1862022114529408/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766591 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766566.923617:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745880 x1862022114529408/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766591 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766566.923644:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766566.923649:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766566.923653:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114529408:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775766566.923661:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114529536:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775766566.923688:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766566.923703:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766566/real 1775766566]  req@ffff9f1485746300 x1862022114529536/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766591 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766566.923736:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746300 x1862022114529536/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766591 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766566.923748:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766566.923752:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766566.923755:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114529536:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775766566.923761:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114529664:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775766566.923781:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766566.923792:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766566/real 1775766566]  req@ffff9f1485746a00 x1862022114529664/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766591 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766566.923803:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746a00 x1862022114529664/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766591 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766566.923810:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766566.923813:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766566.923815:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114529664:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766566.923819:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114529792:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775766566.923841:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766566.923854:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766566/real 1775766566]  req@ffff9f1485744380 x1862022114529792/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766591 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766566.923870:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744380 x1862022114529792/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766591 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766566.923897:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766566.923902:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766566.923906:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114529792:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766566.923931:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114529920:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775766566.923970:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766566.923984:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766566/real 1775766566]  req@ffff9f1485744e00 x1862022114529920/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766591 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766566.924001:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744e00 x1862022114529920/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766591 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766566.924023:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766566.924028:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766566.924033:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114529920:192.168.202.110@tcp:8:kworker.0
00010000:00010000:3.0:1775766567.178732:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766567.946661:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775766568.202710:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766568.970685:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766569.226790:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766569.994664:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766570.250769:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766571.018655:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766571.274825:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766572.042662:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775766572.042674:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766572.042688:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775766572.042695:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766572.042726:0:8:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3949
00000100:00080000:3.0:1775766572.042743:0:8:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766572.042752:0:8:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775766572.042805:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747b80 x1862022114530048/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766572.042858:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766572.042863:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775766572.042865:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766572.042872:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3949
00000100:00080000:3.0:1775766572.042878:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766572.042884:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766572.042898:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766572.042900:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114530048:192.168.202.110@tcp:250:kworker.0
00000100:00100000:3.0:1775766572.042930:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744e00 x1862022114530176/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766572.042950:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766572.042954:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775766572.042957:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766572.042961:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3949
00000100:00080000:3.0:1775766572.042969:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766572.042974:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766572.042981:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000800:00000100:2.0:1775766572.042987:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766572.042997:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744380 x1862022114530304/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766572.043010:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766572.043014:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775766572.043016:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766572.043021:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3949
00000100:00080000:3.0:1775766572.043024:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766572.043028:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766572.043035:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766572.043047:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746a00 x1862022114530432/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775766572.043048:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766572/real 1775766572]  req@ffff9f1485747b80 x1862022114530048/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766597 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766572.043078:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766572.043097:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775766572.043105:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766572.043111:0:8:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3949
00000100:00080000:3.0:1775766572.043117:0:8:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766572.043122:0:8:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766572.043127:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775766572.043128:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747b80 x1862022114530048/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766597 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766572.043144:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746300 x1862022114530560/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766572.043146:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766572.043159:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:3.0:1775766572.043165:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:2.0:1775766572.043171:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114530048:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775766572.043173:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775766572.043181:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766572.043187:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3949
00000100:00080000:3.0:1775766572.043191:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:2.0:1775766572.043207:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114530176:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775766572.043229:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775766572.043235:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766572.043240:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766572.043243:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766572/real 1775766572]  req@ffff9f1485744e00 x1862022114530176/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766597 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766572.043254:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745880 x1862022114530688/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775766572.043259:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744e00 x1862022114530176/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766597 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766572.043302:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766572.043304:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766572.043315:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114530176:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775766572.043318:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:2.0:1775766572.043324:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114530304:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775766572.043326:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775766572.043328:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766572.043332:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3949
00000100:00080000:3.0:1775766572.043336:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766572.043359:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766572.043365:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766572.043378:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747480 x1862022114530816/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:2.0:1775766572.043400:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775766572.043416:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:2.0:1775766572.043419:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766572/real 1775766572]  req@ffff9f1485744380 x1862022114530304/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766597 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766572.043421:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775766572.043426:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766572.043435:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3949
00000100:00080000:3.0:1775766572.043447:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:2.0:1775766572.043470:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744380 x1862022114530304/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766597 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766572.043476:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766572.043493:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00080000:2.0:1775766572.043494:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766572.043507:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766572.043510:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114530304:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766572.043517:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744a80 x1862022114530944/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775766572.043524:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114530432:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775766572.043590:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766572.043601:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766572/real 1775766572]  req@ffff9f1485746a00 x1862022114530432/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766597 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766572.043609:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746a00 x1862022114530432/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766597 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766572.043620:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766572.043623:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766572.043625:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114530432:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775766572.043640:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114530560:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775766572.043653:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766572.043659:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766572/real 1775766572]  req@ffff9f1485746300 x1862022114530560/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766597 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766572.043667:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746300 x1862022114530560/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766597 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766572.043672:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766572.043674:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766572.043675:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114530560:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775766572.043691:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114530688:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775766572.043702:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766572.043708:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766572/real 1775766572]  req@ffff9f1485745880 x1862022114530688/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766597 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766572.043716:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745880 x1862022114530688/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766597 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766572.043721:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766572.043723:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766572.043725:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114530688:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766572.043730:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114530816:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775766572.043740:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766572.043746:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766572/real 1775766572]  req@ffff9f1485747480 x1862022114530816/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766597 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766572.043753:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747480 x1862022114530816/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766597 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766572.043757:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766572.043759:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766572.043761:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114530816:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766572.043765:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114530944:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775766572.043775:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766572.043780:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766572/real 1775766572]  req@ffff9f1485744a80 x1862022114530944/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766597 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766572.043787:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744a80 x1862022114530944/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766597 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766572.043791:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766572.043794:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766572.043797:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114530944:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775766572.298878:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766573.066642:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766573.322749:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766574.090657:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775766574.346756:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766575.114657:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:0.0:1775766575.370824:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766576.138703:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766576.394767:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1775766577.162760:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766577.162771:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00100000:2.0:1775766577.162772:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775766577.162777:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766577.162795:0:8:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3954
00000100:00080000:3.0:1775766577.162811:0:8:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766577.162818:0:8:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775766577.162861:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746680 x1862022114531072/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766577.162895:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766577.162899:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775766577.162901:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766577.162906:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3954
00000100:00080000:3.0:1775766577.162909:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766577.162915:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766577.162926:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766577.162938:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744700 x1862022114531200/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775766577.162946:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114531072:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775766577.162973:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766577.162976:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775766577.162978:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766577.162983:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3954
00000100:00080000:3.0:1775766577.162986:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766577.162990:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766577.162997:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766577.163007:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745500 x1862022114531328/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766577.163017:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766577.163022:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775766577.163023:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766577.163027:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3954
00000100:00080000:3.0:1775766577.163059:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766577.163065:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766577.163071:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:2.0:1775766577.163076:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766577.163089:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744000 x1862022114531456/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766577.163103:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766577.163107:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775766577.163108:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766577.163113:0:8:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3954
00000100:00080000:3.0:1775766577.163118:0:8:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766577.163123:0:8:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766577.163130:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775766577.163139:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766577/real 1775766577]  req@ffff9f1485746680 x1862022114531072/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766602 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766577.163146:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747800 x1862022114531584/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766577.163162:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766577.163175:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775766577.163182:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766577.163191:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3954
00000100:00080000:3.0:1775766577.163212:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:2.0:1775766577.163213:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746680 x1862022114531072/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766602 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766577.163229:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00080000:2.0:1775766577.163246:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000008:00100000:3.0:1775766577.163246:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00080000:2.0:1775766577.163265:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766577.163271:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114531072:192.168.202.110@tcp:250:kworker.0
00000100:00100000:3.0:1775766577.163275:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745c00 x1862022114531712/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766577.163315:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766577.163318:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775766577.163321:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766577.163325:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3954
00000100:00080000:3.0:1775766577.163331:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:2.0:1775766577.163336:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114531200:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775766577.163337:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766577.163352:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766577.163368:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745180 x1862022114531840/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:2.0:1775766577.163381:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775766577.163382:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766577.163391:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775766577.163393:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766577.163399:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3954
00000100:00080000:3.0:1775766577.163405:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766577.163410:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775766577.163416:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766577/real 1775766577]  req@ffff9f1485744700 x1862022114531200/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766602 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:3.0:1775766577.163418:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766577.163442:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745f80 x1862022114531968/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775766577.163445:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744700 x1862022114531200/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766602 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766577.163476:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766577.163480:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766577.163487:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114531200:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766577.163494:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114531328:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775766577.163521:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766577.163531:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766577/real 1775766577]  req@ffff9f1485745500 x1862022114531328/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766602 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766577.163540:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745500 x1862022114531328/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766602 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766577.163548:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766577.163551:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766577.163611:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114531328:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766577.163618:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114531456:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775766577.163641:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766577.163661:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766577/real 1775766577]  req@ffff9f1485744000 x1862022114531456/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766602 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766577.163671:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744000 x1862022114531456/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766602 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766577.163678:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766577.163681:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766577.163700:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114531456:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775766577.163708:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114531584:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775766577.163729:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766577.163738:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766577/real 1775766577]  req@ffff9f1485747800 x1862022114531584/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766602 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766577.163752:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747800 x1862022114531584/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766602 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766577.163760:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766577.163763:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766577.163766:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114531584:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775766577.163772:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114531712:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775766577.163789:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766577.163798:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766577/real 1775766577]  req@ffff9f1485745c00 x1862022114531712/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766602 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766577.163821:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745c00 x1862022114531712/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766602 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766577.163829:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766577.163832:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766577.163835:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114531712:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766577.163859:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114531840:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775766577.163876:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766577.163886:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766577/real 1775766577]  req@ffff9f1485745180 x1862022114531840/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766602 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766577.163898:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745180 x1862022114531840/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766602 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766577.163914:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766577.163918:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766577.163922:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114531840:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766577.163927:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114531968:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775766577.163943:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766577.163952:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766577/real 1775766577]  req@ffff9f1485745f80 x1862022114531968/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766602 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766577.163963:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745f80 x1862022114531968/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766602 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766577.163972:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766577.163975:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766577.163978:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114531968:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775766577.418741:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766578.186724:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766578.442840:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766579.210644:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766579.466727:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766580.234659:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766580.490750:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766581.258683:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775766581.514761:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766582.282694:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:1.0:1775766582.282755:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766582.282765:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1775766582.282769:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766582.282780:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3959
00000100:00080000:1.0:1775766582.282792:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766582.282799:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775766582.282861:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fc380 x1862022114532096/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766582.282893:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766582.282897:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775766582.282899:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766582.282903:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3959
00000100:00080000:1.0:1775766582.282907:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766582.282938:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766582.282948:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775766582.282961:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fe300 x1862022114532224/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766582.282974:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766582.283005:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775766582.283007:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766582.283016:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3959
00000100:00100000:2.0:1775766582.283023:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114532096:192.168.202.110@tcp:250:kworker.0
00000100:00080000:1.0:1775766582.283039:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766582.283049:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766582.283057:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775766582.283079:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fd880 x1862022114532352/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766582.283095:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766582.283100:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775766582.283103:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766582.283111:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3959
00000100:00080000:1.0:1775766582.283116:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766582.283125:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000800:00000100:2.0:1775766582.283127:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000008:00100000:1.0:1775766582.283137:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775766582.283157:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff100 x1862022114532480/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766582.283168:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766582.283181:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1775766582.283183:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766582.283190:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3959
00000100:00100000:2.0:1775766582.283196:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766582/real 1775766582]  req@ffff9f14ab4fc380 x1862022114532096/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766607 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766582.283199:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766582.283211:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766582.283248:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775766582.283274:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fc380 x1862022114532096/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766607 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775766582.283276:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fce00 x1862022114532608/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766582.283353:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775766582.283357:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766582.283363:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:1.0:1775766582.283365:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775766582.283370:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:2.0:1775766582.283377:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114532096:192.168.202.110@tcp:250:kworker.0
00000100:00080000:1.0:1775766582.283384:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3959
00000100:00080000:1.0:1775766582.283392:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766582.283397:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766582.283403:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775766582.283418:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fca80 x1862022114532736/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775766582.283420:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114532224:192.168.202.110@tcp:38:kworker.0
00000100:00080000:1.0:1775766582.283445:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766582.283449:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775766582.283452:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766582.283457:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3959
00000100:00080000:1.0:1775766582.283464:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766582.283469:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766582.283473:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000800:00000100:2.0:1775766582.283479:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775766582.283497:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fed80 x1862022114532864/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775766582.283506:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766582/real 1775766582]  req@ffff9f14ab4fe300 x1862022114532224/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766607 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766582.283521:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766582.283544:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00100000:2.0:1775766582.283608:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fe300 x1862022114532224/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766607 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766582.283616:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766582.283621:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775766582.283626:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3959
00000100:00080000:2.0:1775766582.283636:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00080000:1.0:1775766582.283639:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:2.0:1775766582.283645:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114532224:192.168.202.110@tcp:38:kworker.0
00000100:00080000:1.0:1775766582.283653:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766582.283678:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775766582.283680:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114532352:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775766582.283721:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdc00 x1862022114532992/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:2.0:1775766582.283729:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766582.283751:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766582/real 1775766582]  req@ffff9f14ab4fd880 x1862022114532352/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766607 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766582.283782:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fd880 x1862022114532352/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766607 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766582.283798:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766582.283802:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766582.283805:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114532352:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766582.283812:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114532480:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775766582.283840:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766582.283848:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766582/real 1775766582]  req@ffff9f14ab4ff100 x1862022114532480/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766607 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766582.283858:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff100 x1862022114532480/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766607 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766582.283864:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766582.283867:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766582.283870:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114532480:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775766582.283876:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114532608:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775766582.283892:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766582.283906:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766582/real 1775766582]  req@ffff9f14ab4fce00 x1862022114532608/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766607 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766582.283935:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fce00 x1862022114532608/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766607 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766582.283946:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766582.283951:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766582.283957:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114532608:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775766582.283998:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114532736:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775766582.284039:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766582.284053:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766582/real 1775766582]  req@ffff9f14ab4fca80 x1862022114532736/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766607 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766582.284066:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fca80 x1862022114532736/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766607 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766582.284075:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766582.284079:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766582.284082:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114532736:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766582.284088:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114532864:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775766582.284107:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766582.284116:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766582/real 1775766582]  req@ffff9f14ab4fed80 x1862022114532864/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766607 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766582.284128:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fed80 x1862022114532864/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766607 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766582.284136:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766582.284139:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766582.284157:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114532864:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766582.284164:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114532992:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775766582.284180:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766582.284189:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766582/real 1775766582]  req@ffff9f14ab4fdc00 x1862022114532992/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766607 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766582.284199:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdc00 x1862022114532992/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766607 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766582.284206:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766582.284209:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766582.284211:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114532992:192.168.202.110@tcp:8:kworker.0
00010000:00010000:0.0:1775766582.538751:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766583.306730:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766583.562848:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766584.330771:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766584.586736:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766585.354735:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766585.610766:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766586.378766:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766586.634793:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766587.402717:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775766587.402814:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766587.402826:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775766587.402834:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766587.402854:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3964
00000100:00080000:3.0:1775766587.402873:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766587.402885:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775766587.402954:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747100 x1862022114533120/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766587.403004:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766587.403012:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775766587.403017:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766587.403034:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3964
00000100:00080000:3.0:1775766587.403044:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00100000:2.0:1775766587.403059:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114533120:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775766587.403059:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766587.403083:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766587.403112:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745f80 x1862022114533248/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766587.403131:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766587.403138:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775766587.403142:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766587.403150:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3964
00000100:00080000:3.0:1775766587.403158:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000800:00000100:2.0:1775766587.403165:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775766587.403170:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766587.403213:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766587.403237:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745180 x1862022114533376/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766587.403255:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766587.403260:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775766587.403264:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:2.0:1775766587.403268:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766587/real 1775766587]  req@ffff9f1485747100 x1862022114533120/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766612 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766587.403276:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3964
00000100:00080000:3.0:1775766587.403288:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766587.403300:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766587.403310:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775766587.403318:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747100 x1862022114533120/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766612 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766587.403333:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745c00 x1862022114533504/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766587.403339:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766587.403351:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766587.403362:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114533120:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775766587.403370:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766587.403378:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775766587.403381:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766587.403389:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3964
00000100:00080000:3.0:1775766587.403397:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00100000:2.0:1775766587.403400:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114533248:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775766587.403410:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000800:00000100:2.0:1775766587.403438:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766587.403453:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766587/real 1775766587]  req@ffff9f1485745f80 x1862022114533248/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766612 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766587.403466:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745f80 x1862022114533248/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766612 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766587.403475:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766587.403479:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766587.403482:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114533248:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766587.403487:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114533376:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775766587.403504:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766587.403512:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766587/real 1775766587]  req@ffff9f1485745180 x1862022114533376/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766612 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766587.403523:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745180 x1862022114533376/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766612 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766587.403531:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766587.403534:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766587.403537:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114533376:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766587.403543:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114533504:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775766587.403595:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766587.403611:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766587/real 1775766587]  req@ffff9f1485745c00 x1862022114533504/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766612 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766587.403628:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745c00 x1862022114533504/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766612 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766587.403639:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766587.403645:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766587.403651:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114533504:192.168.202.110@tcp:8:kworker.0
00000008:00100000:3.0:1775766587.403846:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766587.403954:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747800 x1862022114533632/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766587.403989:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766587.403996:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775766587.403999:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766587.404006:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3964
00000100:00080000:3.0:1775766587.404014:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766587.404022:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766587.404032:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766587.404053:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744000 x1862022114533760/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766587.404070:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766587.404075:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775766587.404079:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:2.0:1775766587.404085:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114533632:192.168.202.110@tcp:8:kworker.0
00000100:00080000:3.0:1775766587.404092:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3964
00000100:00080000:3.0:1775766587.404101:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766587.404108:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766587.404118:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000800:00000100:2.0:1775766587.404132:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766587.404143:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745500 x1862022114533888/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775766587.404156:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766587/real 1775766587]  req@ffff9f1485747800 x1862022114533632/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766612 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766587.404177:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766587.404206:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00100000:2.0:1775766587.404211:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747800 x1862022114533632/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766612 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766587.404212:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766587.404229:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3964
00000100:00080000:2.0:1775766587.404233:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766587.404239:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766587.404245:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00080000:3.0:1775766587.404252:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775766587.404259:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114533632:192.168.202.110@tcp:8:kworker.0
00000008:00100000:3.0:1775766587.404268:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775766587.404275:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114533760:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766587.404294:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744700 x1862022114534016/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:2.0:1775766587.404322:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766587.404344:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766587/real 1775766587]  req@ffff9f1485744000 x1862022114533760/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766612 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766587.404361:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744000 x1862022114533760/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766612 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766587.404377:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766587.404381:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766587.404384:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114533760:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766587.404401:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114533888:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775766587.404424:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766587.404434:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766587/real 1775766587]  req@ffff9f1485745500 x1862022114533888/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766612 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766587.404446:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745500 x1862022114533888/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766612 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766587.404454:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766587.404458:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766587.404461:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114533888:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766587.404466:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114534016:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775766587.404480:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766587.404488:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766587/real 1775766587]  req@ffff9f1485744700 x1862022114534016/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766612 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766587.404499:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744700 x1862022114534016/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766612 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766587.404506:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766587.404509:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766587.404513:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114534016:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775766587.658848:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766588.426622:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775766588.682793:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766589.450684:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:0.0:1775766589.706762:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766590.474656:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766590.730719:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766591.498647:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766591.754803:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1775766592.522703:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766592.522716:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775766592.522723:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00100000:2.0:1775766592.522742:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775766592.522765:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3970
00000100:00080000:3.0:1775766592.522782:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766592.522792:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775766592.522851:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746680 x1862022114534144/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766592.522901:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766592.522906:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775766592.522908:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766592.522913:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3970
00000100:00080000:3.0:1775766592.522918:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766592.522923:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766592.522936:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766592.522951:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114534144:192.168.202.110@tcp:250:kworker.0
00000100:00100000:3.0:1775766592.522955:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744a80 x1862022114534272/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766592.522972:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766592.522976:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775766592.522978:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766592.522983:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3970
00000100:00080000:3.0:1775766592.522988:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766592.522991:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766592.522996:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766592.523010:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747480 x1862022114534400/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766592.523019:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766592.523023:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775766592.523025:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766592.523029:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3970
00000100:00080000:3.0:1775766592.523034:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766592.523038:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766592.523043:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766592.523060:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745880 x1862022114534528/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:2.0:1775766592.523061:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775766592.523075:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766592.523079:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775766592.523082:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766592.523088:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3970
00000100:00080000:3.0:1775766592.523094:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766592.523098:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766592.523102:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766592.523113:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746300 x1862022114534656/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766592.523151:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:2.0:1775766592.523155:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766592/real 1775766592]  req@ffff9f1485746680 x1862022114534144/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766617 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766592.523159:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775766592.523165:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766592.523191:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3970
00000100:00080000:3.0:1775766592.523203:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:2.0:1775766592.523214:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746680 x1862022114534144/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766617 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766592.523215:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766592.523235:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00080000:2.0:1775766592.523237:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766592.523249:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766592.523255:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114534144:192.168.202.110@tcp:250:kworker.0
00000100:00100000:3.0:1775766592.523263:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746a00 x1862022114534784/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766592.523276:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766592.523280:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775766592.523283:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766592.523289:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3970
00000100:00100000:2.0:1775766592.523294:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114534272:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775766592.523298:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766592.523307:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766592.523315:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000800:00000100:2.0:1775766592.523345:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766592.523347:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744380 x1862022114534912/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766592.523363:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:2.0:1775766592.523368:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766592/real 1775766592]  req@ffff9f1485744a80 x1862022114534272/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766617 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766592.523370:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775766592.523376:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766592.523390:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3970
00000100:00080000:3.0:1775766592.523403:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766592.523408:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775766592.523431:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744a80 x1862022114534272/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766617 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:3.0:1775766592.523436:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00080000:2.0:1775766592.523462:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00100000:3.0:1775766592.523465:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744e00 x1862022114535040/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766592.523472:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766592.523500:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114534272:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766592.523518:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114534400:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775766592.523593:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766592.523608:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766592/real 1775766592]  req@ffff9f1485747480 x1862022114534400/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766617 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766592.523622:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747480 x1862022114534400/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766617 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766592.523631:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766592.523635:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766592.523638:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114534400:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766592.523644:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114534528:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775766592.523666:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766592.523676:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766592/real 1775766592]  req@ffff9f1485745880 x1862022114534528/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766617 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766592.523689:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745880 x1862022114534528/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766617 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766592.523697:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766592.523700:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766592.523704:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114534528:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775766592.523710:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114534656:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775766592.523729:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766592.523745:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766592/real 1775766592]  req@ffff9f1485746300 x1862022114534656/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766617 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766592.523756:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746300 x1862022114534656/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766617 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766592.523765:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766592.523768:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766592.523771:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114534656:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775766592.523776:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114534784:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775766592.523792:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766592.523801:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766592/real 1775766592]  req@ffff9f1485746a00 x1862022114534784/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766617 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766592.523812:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746a00 x1862022114534784/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766617 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766592.523820:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766592.523824:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766592.523827:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114534784:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766592.523848:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114534912:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775766592.523865:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766592.523886:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766592/real 1775766592]  req@ffff9f1485744380 x1862022114534912/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766617 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766592.523898:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744380 x1862022114534912/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766617 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766592.523907:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766592.523911:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766592.523914:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114534912:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766592.523919:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114535040:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775766592.523934:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766592.523942:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766592/real 1775766592]  req@ffff9f1485744e00 x1862022114535040/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766617 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766592.523955:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744e00 x1862022114535040/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766617 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766592.523963:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766592.523966:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766592.523969:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114535040:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775766592.778876:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766593.546638:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766593.802827:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766594.570641:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766594.826731:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766595.594647:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775766595.850758:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766596.618642:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:0.0:1775766596.874774:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766597.642683:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:0.0:1775766597.642743:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766597.642756:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1775766597.642762:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766597.642775:0:8:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3975
00000100:00080000:0.0:1775766597.642789:0:8:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766597.642808:0:8:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775766597.642855:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3100 x1862022114535168/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775766597.642906:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766597.642911:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775766597.642914:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766597.642919:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3975
00000100:00080000:0.0:1775766597.642923:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766597.642928:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775766597.642940:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766597.642964:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114535168:192.168.202.110@tcp:250:kworker.0
00000100:00100000:0.0:1775766597.642985:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d2a00 x1862022114535296/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775766597.643006:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766597.643012:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:0.0:1775766597.643014:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766597.643022:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3975
00000100:00080000:0.0:1775766597.643026:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766597.643030:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775766597.643039:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775766597.643054:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3b80 x1862022114535424/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775766597.643067:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766597.643071:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775766597.643102:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766597.643113:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3975
00000100:00080000:0.0:1775766597.643133:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766597.643157:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775766597.643166:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775766597.643172:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114535296:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766597.643225:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0a80 x1862022114535552/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775766597.643236:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114535424:192.168.202.110@tcp:38:kworker.0
00000100:00080000:0.0:1775766597.643261:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766597.643267:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1775766597.643270:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766597.643287:0:8:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3975
00000100:00080000:0.0:1775766597.643296:0:8:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766597.643305:0:8:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775766597.643314:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775766597.643314:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114535552:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775766597.643342:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d1f80 x1862022114535680/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775766597.643359:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766597.643363:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775766597.643368:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:2.0:1775766597.643369:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114535680:192.168.202.110@tcp:8:kworker.0
00000100:00080000:0.0:1775766597.643389:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3975
00000100:00080000:0.0:1775766597.643396:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766597.643406:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775766597.643416:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775766597.643435:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0380 x1862022114535808/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775766597.643452:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766597.643456:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:0.0:1775766597.643459:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766597.643464:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3975
00000100:00080000:0.0:1775766597.643473:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766597.643479:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775766597.643485:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114535808:192.168.202.110@tcp:38:kworker.0
00000008:00100000:0.0:1775766597.643486:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775766597.643513:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0e00 x1862022114535936/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775766597.643529:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766597.643533:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775766597.643535:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:2.0:1775766597.643538:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114535936:192.168.202.110@tcp:38:kworker.0
00000100:00080000:0.0:1775766597.643545:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3975
00000100:00080000:0.0:1775766597.643640:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766597.643650:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775766597.643662:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775766597.643686:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d2300 x1862022114536064/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775766597.643791:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114536064:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775766597.898846:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766598.666686:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766598.922674:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766599.690640:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766599.946693:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766600.714644:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000400:00000100:1.0:1775766600.714811:0:2393:0:(lib-socket.c:454:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.202.110:988
00000400:00000100:1.0:1775766600.714845:0:2393:0:(acceptor.c:111:lnet_connect_console_error()) Connection to 192.168.202.110@tcp at host 192.168.202.110 was unreachable: the network or that node may be down, or Lustre may be misconfigured.
00000800:00000100:1.0:1775766600.714862:0:2393:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:1.0:1775766600.714912:0:2393:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:1.0:1775766600.714926:0:2393:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:1.0:1775766600.714939:0:2393:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:1.0:1775766600.714951:0:2393:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:1.0:1775766600.714963:0:2393:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000100:00100000:2.0:1775766600.714964:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766597/real 1775766600]  req@ffff9f14a04d0a80 x1862022114535552/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766622 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000800:00000100:1.0:1775766600.714977:0:2393:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:1.0:1775766600.714992:0:2393:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000100:00100000:2.0:1775766600.715006:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0a80 x1862022114535552/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766622 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766600.715031:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766600.715037:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766600.715068:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114535552:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775766600.715081:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766597/real 1775766600]  req@ffff9f14a04d2a00 x1862022114535296/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766622 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766600.715099:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d2a00 x1862022114535296/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766622 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766600.715107:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766600.715111:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766600.715114:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114535296:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766600.715122:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766597/real 1775766600]  req@ffff9f14a04d3b80 x1862022114535424/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766622 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766600.715136:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3b80 x1862022114535424/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766622 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766600.715145:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766600.715149:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766600.715152:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114535424:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766600.715160:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766597/real 1775766600]  req@ffff9f14a04d3100 x1862022114535168/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766622 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766600.715179:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3100 x1862022114535168/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766622 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766600.715188:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766600.715192:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766600.715195:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114535168:192.168.202.110@tcp:250:kworker.0
00000100:00100000:2.0:1775766600.715226:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766597/real 1775766600]  req@ffff9f14a04d2300 x1862022114536064/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766622 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766600.715240:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d2300 x1862022114536064/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766622 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766600.715248:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766600.715252:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766600.715255:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114536064:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775766600.715261:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766597/real 1775766600]  req@ffff9f14a04d0e00 x1862022114535936/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766622 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766600.715273:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0e00 x1862022114535936/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766622 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766600.715281:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766600.715284:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766600.715287:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114535936:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766600.715295:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766597/real 1775766600]  req@ffff9f14a04d0380 x1862022114535808/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766622 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766600.715305:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0380 x1862022114535808/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766622 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766600.715313:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766600.715316:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766600.715319:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114535808:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766600.715326:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766597/real 1775766600]  req@ffff9f14a04d1f80 x1862022114535680/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766622 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766600.715337:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1f80 x1862022114535680/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766622 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766600.715344:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766600.715348:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766600.715352:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114535680:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775766600.970712:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766601.738607:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766601.994763:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766602.762772:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775766602.762774:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766602.762790:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775766602.762797:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766602.762811:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3980
00000100:00080000:3.0:1775766602.762828:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766602.762838:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775766602.762889:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747b80 x1862022114536192/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766602.762942:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766602.762947:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775766602.762950:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766602.762956:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3980
00000100:00080000:3.0:1775766602.762961:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766602.762968:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766602.762984:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766602.763002:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114536192:192.168.202.110@tcp:250:kworker.0
00000100:00100000:3.0:1775766602.763014:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744e00 x1862022114536320/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766602.763051:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766602.763057:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775766602.763060:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766602.763066:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3980
00000100:00080000:3.0:1775766602.763083:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766602.763088:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766602.763095:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766602.763114:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744380 x1862022114536448/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766602.763128:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766602.763133:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775766602.763136:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766602.763144:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3980
00000800:00000100:2.0:1775766602.763146:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775766602.763155:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766602.763163:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766602.763170:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766602.763187:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746a00 x1862022114536576/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766602.763202:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766602.763207:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775766602.763211:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766602.763216:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3980
00000100:00080000:3.0:1775766602.763222:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00100000:2.0:1775766602.763230:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766602/real 1775766602]  req@ffff9f1485747b80 x1862022114536192/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766627 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766602.763234:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766602.763247:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766602.763271:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746300 x1862022114536704/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775766602.763281:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747b80 x1862022114536192/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766627 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766602.763298:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766602.763315:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775766602.763323:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766602.763339:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3980
00000100:00080000:3.0:1775766602.763346:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766602.763347:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766602.763359:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00080000:2.0:1775766602.763366:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000008:00100000:3.0:1775766602.763371:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766602.763381:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114536192:192.168.202.110@tcp:250:kworker.0
00000100:00100000:3.0:1775766602.763390:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745880 x1862022114536832/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766602.763416:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766602.763422:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775766602.763427:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:2.0:1775766602.763429:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114536320:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775766602.763438:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3980
00000100:00080000:3.0:1775766602.763451:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766602.763457:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766602.763464:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766602.763481:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747480 x1862022114536960/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:2.0:1775766602.763495:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766602.763518:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766602/real 1775766602]  req@ffff9f1485744e00 x1862022114536320/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766627 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766602.763520:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766602.763532:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775766602.763538:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766602.763552:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3980
00000100:00100000:2.0:1775766602.763606:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744e00 x1862022114536320/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766627 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766602.763619:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766602.763624:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766602.763629:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00080000:3.0:1775766602.763639:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775766602.763645:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114536320:192.168.202.110@tcp:38:kworker.0
00000008:00100000:3.0:1775766602.763653:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775766602.763660:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114536448:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766602.763681:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744a80 x1862022114537088/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:2.0:1775766602.763700:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766602.763722:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766602/real 1775766602]  req@ffff9f1485744380 x1862022114536448/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766627 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766602.763742:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744380 x1862022114536448/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766627 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766602.763751:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766602.763755:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766602.763759:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114536448:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766602.763768:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114536576:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775766602.763795:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766602.763812:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766602/real 1775766602]  req@ffff9f1485746a00 x1862022114536576/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766627 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766602.763825:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746a00 x1862022114536576/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766627 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766602.763836:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766602.763841:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766602.763844:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114536576:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775766602.763852:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114536704:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775766602.763875:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766602.763889:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766602/real 1775766602]  req@ffff9f1485746300 x1862022114536704/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766627 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766602.763905:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746300 x1862022114536704/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766627 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766602.763919:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766602.763924:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766602.763928:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114536704:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775766602.763936:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114536832:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775766602.763960:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766602.763974:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766602/real 1775766602]  req@ffff9f1485745880 x1862022114536832/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766627 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766602.763989:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745880 x1862022114536832/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766627 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766602.763998:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766602.764002:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766602.764006:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114536832:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766602.764051:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114536960:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775766602.764076:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766602.764089:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766602/real 1775766602]  req@ffff9f1485747480 x1862022114536960/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766627 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766602.764103:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747480 x1862022114536960/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766627 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766602.764112:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766602.764116:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766602.764120:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114536960:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766602.764129:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114537088:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775766602.764151:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766602.764163:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766602/real 1775766602]  req@ffff9f1485744a80 x1862022114537088/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766627 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766602.764176:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744a80 x1862022114537088/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766627 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766602.764185:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766602.764189:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766602.764192:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114537088:192.168.202.110@tcp:8:kworker.0
00010000:00010000:3.0:1775766603.018847:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766603.786678:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775766604.042692:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766604.810693:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766605.066888:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766605.834682:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766606.090782:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766606.858697:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766607.114744:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766607.882736:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:1.0:1775766607.882772:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766607.882781:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1775766607.882788:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766607.882803:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3985
00000100:00080000:1.0:1775766607.882822:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766607.882833:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775766607.882904:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ffb80 x1862022114537216/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766607.882944:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766607.882952:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775766607.882955:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766607.882964:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3985
00000100:00080000:1.0:1775766607.882991:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766607.882999:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766607.883034:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766607.883053:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ffb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114537216:192.168.202.110@tcp:250:kworker.0
00000100:00100000:1.0:1775766607.883065:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fea00 x1862022114537344/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766607.883080:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766607.883085:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775766607.883088:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766607.883094:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3985
00000100:00080000:1.0:1775766607.883099:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766607.883103:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766607.883110:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775766607.883138:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff800 x1862022114537472/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:2.0:1775766607.883144:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:1.0:1775766607.883164:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766607.883168:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775766607.883171:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766607.883176:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3985
00000100:00080000:1.0:1775766607.883181:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766607.883186:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766607.883194:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775766607.883209:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdf80 x1862022114537600/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775766607.883217:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766607/real 1775766607]  req@ffff9f14ab4ffb80 x1862022114537216/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766632 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766607.883225:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766607.883230:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1775766607.883233:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766607.883238:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3985
00000100:00080000:1.0:1775766607.883242:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00100000:2.0:1775766607.883244:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ffb80 x1862022114537216/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766632 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766607.883248:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766607.883255:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00080000:2.0:1775766607.883261:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766607.883267:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775766607.883270:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdc00 x1862022114537728/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775766607.883274:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ffb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114537216:192.168.202.110@tcp:250:kworker.0
00000100:00080000:1.0:1775766607.883284:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766607.883289:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775766607.883292:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766607.883297:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3985
00000100:00080000:1.0:1775766607.883302:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:2.0:1775766607.883307:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114537344:192.168.202.110@tcp:38:kworker.0
00000100:00080000:1.0:1775766607.883308:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766607.883315:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775766607.883329:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fed80 x1862022114537856/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:2.0:1775766607.883358:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:1.0:1775766607.883362:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766607.883367:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775766607.883370:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766607.883376:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3985
00000100:00100000:2.0:1775766607.883377:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766607/real 1775766607]  req@ffff9f14ab4fea00 x1862022114537344/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766632 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766607.883383:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:2.0:1775766607.883398:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fea00 x1862022114537344/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766632 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766607.883404:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766607.883413:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00080000:2.0:1775766607.883413:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766607.883419:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766607.883424:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114537344:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775766607.883430:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fca80 x1862022114537984/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775766607.883435:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114537472:192.168.202.110@tcp:38:kworker.0
00000100:00080000:1.0:1775766607.883446:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766607.883463:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775766607.883466:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766607.883472:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3985
00000800:00000100:2.0:1775766607.883477:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:1.0:1775766607.883478:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766607.883483:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766607.883492:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775766607.883493:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766607/real 1775766607]  req@ffff9f14ab4ff800 x1862022114537472/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766632 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775766607.883508:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fce00 x1862022114538112/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775766607.883513:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff800 x1862022114537472/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766632 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766607.883527:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766607.883532:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766607.883538:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114537472:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766607.883548:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114537600:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775766607.883597:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766607.883644:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766607/real 1775766607]  req@ffff9f14ab4fdf80 x1862022114537600/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766632 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766607.883658:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdf80 x1862022114537600/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766632 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766607.883667:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766607.883671:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766607.883674:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114537600:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775766607.883680:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114537728:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775766607.883710:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766607.883720:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766607/real 1775766607]  req@ffff9f14ab4fdc00 x1862022114537728/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766632 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766607.883732:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdc00 x1862022114537728/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766632 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766607.883739:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766607.883742:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766607.883745:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114537728:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775766607.883765:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114537856:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775766607.883781:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766607.883790:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766607/real 1775766607]  req@ffff9f14ab4fed80 x1862022114537856/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766632 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766607.883802:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fed80 x1862022114537856/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766632 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766607.883809:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766607.883812:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766607.883815:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114537856:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766607.883820:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114537984:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775766607.883834:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766607.883842:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766607/real 1775766607]  req@ffff9f14ab4fca80 x1862022114537984/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766632 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766607.883853:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fca80 x1862022114537984/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766632 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766607.883867:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766607.883870:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766607.883873:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114537984:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766607.883900:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114538112:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775766607.883919:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766607.883928:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766607/real 1775766607]  req@ffff9f14ab4fce00 x1862022114538112/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766632 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766607.883940:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fce00 x1862022114538112/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766632 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766607.883949:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766607.883952:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766607.883955:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114538112:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775766608.138823:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766608.906652:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766609.162760:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766609.930734:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775766610.186716:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766610.954650:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775766611.210733:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766611.978635:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766612.234690:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766613.002634:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775766613.002653:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766613.002659:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775766613.002664:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766613.002673:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3990
00000100:00080000:3.0:1775766613.002683:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766613.002689:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775766613.002723:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746680 x1862022114538240/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766613.002747:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766613.002750:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775766613.002752:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766613.002755:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3990
00000100:00080000:3.0:1775766613.002758:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766613.002761:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766613.002769:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766613.002779:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114538240:192.168.202.110@tcp:250:kworker.0
00000100:00100000:3.0:1775766613.002780:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744700 x1862022114538368/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766613.002792:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766613.002795:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775766613.002796:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766613.002799:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3990
00000100:00080000:3.0:1775766613.002801:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766613.002803:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766613.002806:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766613.002813:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745500 x1862022114538496/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766613.002823:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766613.002826:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775766613.002827:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766613.002829:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3990
00000100:00080000:3.0:1775766613.002838:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766613.002841:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766613.002844:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766613.002851:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744000 x1862022114538624/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:2.0:1775766613.002857:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775766613.002861:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766613.002864:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775766613.002865:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766613.002869:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3990
00000100:00080000:3.0:1775766613.002871:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766613.002874:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766613.002876:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766613.002883:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747800 x1862022114538752/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766613.002889:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766613.002891:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775766613.002892:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766613.002894:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3990
00000100:00080000:3.0:1775766613.002898:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766613.002902:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766613.002906:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766613.002911:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766613/real 1775766613]  req@ffff9f1485746680 x1862022114538240/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766638 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766613.002931:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745c00 x1862022114538880/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766613.002948:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766613.002950:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775766613.002958:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:2.0:1775766613.002961:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746680 x1862022114538240/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766638 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766613.002974:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3990
00000100:00080000:3.0:1775766613.002980:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766613.002985:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00080000:2.0:1775766613.002988:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000008:00100000:3.0:1775766613.002989:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00080000:2.0:1775766613.002994:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766613.003001:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114538240:192.168.202.110@tcp:250:kworker.0
00000100:00100000:3.0:1775766613.003001:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745180 x1862022114539008/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766613.003014:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766613.003017:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775766613.003018:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766613.003021:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3990
00000100:00080000:3.0:1775766613.003023:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766613.003025:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775766613.003029:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114538368:192.168.202.110@tcp:38:kworker.0
00000008:00100000:3.0:1775766613.003031:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766613.003042:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745f80 x1862022114539136/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:2.0:1775766613.003055:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766613.003067:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766613/real 1775766613]  req@ffff9f1485744700 x1862022114538368/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766638 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766613.003078:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744700 x1862022114538368/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766638 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766613.003087:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766613.003091:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766613.003094:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114538368:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766613.003101:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114538496:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775766613.003121:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766613.003133:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766613/real 1775766613]  req@ffff9f1485745500 x1862022114538496/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766638 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766613.003146:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745500 x1862022114538496/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766638 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766613.003157:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766613.003160:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766613.003162:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114538496:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766613.003167:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114538624:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775766613.003183:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766613.003190:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766613/real 1775766613]  req@ffff9f1485744000 x1862022114538624/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766638 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766613.003197:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744000 x1862022114538624/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766638 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766613.003203:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766613.003205:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766613.003207:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114538624:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775766613.003211:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114538752:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775766613.003220:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766613.003225:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766613/real 1775766613]  req@ffff9f1485747800 x1862022114538752/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766638 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766613.003232:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747800 x1862022114538752/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766638 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766613.003239:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766613.003242:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766613.003246:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114538752:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775766613.003252:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114538880:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775766613.003271:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766613.003292:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766613/real 1775766613]  req@ffff9f1485745c00 x1862022114538880/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766638 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766613.003305:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745c00 x1862022114538880/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766638 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766613.003311:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766613.003313:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766613.003315:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114538880:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766613.003319:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114539008:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775766613.003335:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766613.003346:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766613/real 1775766613]  req@ffff9f1485745180 x1862022114539008/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766638 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766613.003357:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745180 x1862022114539008/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766638 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766613.003365:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766613.003369:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766613.003372:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114539008:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766613.003393:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114539136:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775766613.003409:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766613.003417:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766613/real 1775766613]  req@ffff9f1485745f80 x1862022114539136/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766638 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766613.003425:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745f80 x1862022114539136/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766638 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766613.003430:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766613.003432:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766613.003434:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114539136:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775766613.258755:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766614.026655:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766614.282704:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766615.050628:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766615.306735:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766616.074725:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766616.330741:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766617.098631:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775766617.354718:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1775766618.122645:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:2.0:1775766618.122653:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775766618.122654:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775766618.122659:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766618.122668:0:8:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 3995
00000100:00080000:3.0:1775766618.122691:0:8:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766618.122699:0:8:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775766618.122741:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747100 x1862022114539264/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766618.122779:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766618.122784:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775766618.122786:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766618.122791:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3995
00000100:00080000:3.0:1775766618.122795:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766618.122801:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766618.122810:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766618.122819:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114539264:192.168.202.110@tcp:250:kworker.0
00000100:00100000:3.0:1775766618.122826:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745f80 x1862022114539392/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766618.122842:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766618.122845:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775766618.122848:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766618.122852:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3995
00000100:00080000:3.0:1775766618.122888:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766618.122894:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000800:00000100:2.0:1775766618.122900:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000008:00100000:3.0:1775766618.122901:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766618.122917:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745180 x1862022114539520/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766618.122927:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766618.122930:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775766618.122931:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766618.122935:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3995
00000100:00100000:2.0:1775766618.122939:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766618/real 1775766618]  req@ffff9f1485747100 x1862022114539264/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766643 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766618.122939:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766618.122947:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766618.122955:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775766618.122963:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747100 x1862022114539264/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766643 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766618.122969:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745c00 x1862022114539648/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766618.122981:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766618.122985:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775766618.122988:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766618.122991:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766618.122991:0:8:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 3995
00000100:00080000:3.0:1775766618.122996:0:8:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766618.122999:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:3.0:1775766618.123002:0:8:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775766618.123006:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114539264:192.168.202.110@tcp:250:kworker.0
00000008:00100000:3.0:1775766618.123006:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766618.123017:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747800 x1862022114539776/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766618.123023:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:2.0:1775766618.123031:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114539392:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775766618.123038:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775766618.123039:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766618.123043:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3995
00000100:00080000:3.0:1775766618.123047:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766618.123050:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766618.123055:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000800:00000100:2.0:1775766618.123056:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766618.123067:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744000 x1862022114539904/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775766618.123071:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766618/real 1775766618]  req@ffff9f1485745f80 x1862022114539392/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766643 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766618.123094:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:2.0:1775766618.123097:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745f80 x1862022114539392/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766643 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766618.123098:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775766618.123102:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766618.123110:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3995
00000100:00080000:3.0:1775766618.123112:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766618.123114:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766618.123121:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00080000:2.0:1775766618.123122:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766618.123128:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766618.123133:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745500 x1862022114540032/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775766618.123134:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114539392:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766618.123145:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114539520:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775766618.123149:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766618.123152:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775766618.123155:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766618.123159:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 3995
00000100:00080000:3.0:1775766618.123163:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000800:00000100:2.0:1775766618.123165:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775766618.123169:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775766618.123175:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766618/real 1775766618]  req@ffff9f1485745180 x1862022114539520/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766643 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:3.0:1775766618.123175:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775766618.123188:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745180 x1862022114539520/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766643 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766618.123195:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744700 x1862022114540160/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766618.123196:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766618.123202:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766618.123208:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114539520:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766618.123218:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114539648:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775766618.123237:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766618.123248:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766618/real 1775766618]  req@ffff9f1485745c00 x1862022114539648/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766643 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766618.123262:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745c00 x1862022114539648/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766643 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766618.123271:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766618.123274:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766618.123278:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114539648:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775766618.123284:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114539776:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775766618.123303:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766618.123311:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766618/real 1775766618]  req@ffff9f1485747800 x1862022114539776/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766643 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766618.123319:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747800 x1862022114539776/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766643 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766618.123324:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766618.123327:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766618.123329:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114539776:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775766618.123342:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114539904:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775766618.123352:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766618.123358:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766618/real 1775766618]  req@ffff9f1485744000 x1862022114539904/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766643 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766618.123365:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744000 x1862022114539904/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766643 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766618.123375:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766618.123377:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766618.123379:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114539904:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766618.123382:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114540032:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775766618.123391:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766618.123396:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766618/real 1775766618]  req@ffff9f1485745500 x1862022114540032/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766643 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766618.123403:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745500 x1862022114540032/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766643 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766618.123408:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766618.123410:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766618.123411:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114540032:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766618.123415:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114540160:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775766618.123425:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766618.123433:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766618/real 1775766618]  req@ffff9f1485744700 x1862022114540160/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766643 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766618.123445:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744700 x1862022114540160/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766643 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766618.123453:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766618.123457:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766618.123461:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114540160:192.168.202.110@tcp:8:kworker.0
00010000:00010000:1.0:1775766618.378696:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766619.146636:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766619.402780:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766620.170712:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766620.426751:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766621.194613:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766621.450693:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766622.218645:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766622.474723:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1775766623.242654:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:2.0:1775766623.242659:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775766623.242662:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775766623.242667:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766623.242677:0:8:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4000
00000100:00080000:3.0:1775766623.242687:0:8:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766623.242695:0:8:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775766623.242754:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746680 x1862022114540288/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766623.242775:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766623.242779:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775766623.242780:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766623.242783:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4000
00000100:00080000:3.0:1775766623.242786:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766623.242789:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766623.242798:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766623.242826:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114540288:192.168.202.110@tcp:250:kworker.0
00000100:00100000:3.0:1775766623.242830:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744a80 x1862022114540416/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766623.242846:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766623.242851:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775766623.242853:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766623.242859:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4000
00000100:00080000:3.0:1775766623.242865:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766623.242870:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766623.242875:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766623.242885:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747480 x1862022114540544/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766623.242892:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766623.242896:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775766623.242897:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766623.242901:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4000
00000100:00080000:3.0:1775766623.242904:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000800:00000100:2.0:1775766623.242906:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775766623.242910:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766623.242915:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766623.242923:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745880 x1862022114540672/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766623.242931:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766623.242934:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775766623.242936:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766623.242938:0:8:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4000
00000100:00080000:3.0:1775766623.242940:0:8:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766623.242943:0:8:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766623.242947:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775766623.242958:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766623/real 1775766623]  req@ffff9f1485746680 x1862022114540288/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766648 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766623.242977:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746680 x1862022114540288/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766648 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766623.242988:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766623.242992:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766623.242997:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114540288:192.168.202.110@tcp:250:kworker.0
00000100:00100000:2.0:1775766623.243019:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114540416:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766623.243044:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746300 x1862022114540800/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:2.0:1775766623.243048:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775766623.243065:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:2.0:1775766623.243067:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766623/real 1775766623]  req@ffff9f1485744a80 x1862022114540416/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766648 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766623.243070:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775766623.243074:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766623.243081:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4000
00000100:00080000:3.0:1775766623.243088:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766623.243092:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775766623.243097:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744a80 x1862022114540416/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766648 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:3.0:1775766623.243097:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00080000:2.0:1775766623.243108:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766623.243112:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766623.243116:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114540416:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766623.243121:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114540544:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775766623.243139:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766623.243147:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766623/real 1775766623]  req@ffff9f1485747480 x1862022114540544/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766648 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766623.243155:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747480 x1862022114540544/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766648 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766623.243161:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766623.243164:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766623.243165:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114540544:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766623.243169:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114540672:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775766623.243191:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766623.243199:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766623/real 1775766623]  req@ffff9f1485745880 x1862022114540672/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766648 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766623.243210:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746a00 x1862022114540928/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775766623.243211:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745880 x1862022114540672/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766648 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766623.243228:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766623.243230:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766623.243232:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114540672:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775766623.243244:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114540800:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775766623.243257:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766623.243264:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766623/real 1775766623]  req@ffff9f1485746300 x1862022114540800/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766648 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766623.243273:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746300 x1862022114540800/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766648 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766623.243278:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766623.243280:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766623.243284:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114540800:192.168.202.110@tcp:8:kworker.0
00000100:00080000:3.0:1775766623.243305:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766623.243308:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775766623.243310:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766623.243314:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4000
00000100:00080000:3.0:1775766623.243318:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766623.243322:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766623.243327:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766623.243331:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114540928:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766623.243338:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744380 x1862022114541056/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766623.243347:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766623.243349:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775766623.243351:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766623.243353:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4000
00000800:00000100:2.0:1775766623.243357:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775766623.243358:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766623.243363:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766623.243368:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775766623.243371:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766623/real 1775766623]  req@ffff9f1485746a00 x1862022114540928/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766648 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766623.243382:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744e00 x1862022114541184/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775766623.243383:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746a00 x1862022114540928/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766648 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766623.243412:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766623.243416:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766623.243419:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114540928:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766623.243429:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114541056:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775766623.243441:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766623.243447:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766623/real 1775766623]  req@ffff9f1485744380 x1862022114541056/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766648 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766623.243455:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744380 x1862022114541056/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766648 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766623.243460:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766623.243462:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766623.243464:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114541056:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766623.243467:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114541184:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775766623.243477:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766623.243482:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766623/real 1775766623]  req@ffff9f1485744e00 x1862022114541184/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766648 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766623.243489:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744e00 x1862022114541184/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766648 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766623.243516:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766623.243518:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766623.243520:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114541184:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775766623.498701:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766624.266647:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775766624.522756:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766625.290635:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775766625.546692:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766626.314665:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766626.570735:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766627.338652:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766627.594738:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766628.362683:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775766628.362773:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766628.362779:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775766628.362782:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766628.362790:0:8:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4005
00000100:00080000:3.0:1775766628.362799:0:8:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766628.362805:0:8:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775766628.362840:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747b80 x1862022114541312/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766628.362864:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766628.362867:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775766628.362868:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766628.362871:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4005
00000100:00080000:3.0:1775766628.362873:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766628.362875:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766628.362883:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766628.362892:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744e00 x1862022114541440/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766628.362900:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766628.362903:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775766628.362905:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766628.362908:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4005
00000100:00080000:3.0:1775766628.362911:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766628.362913:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766628.362916:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766628.362921:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114541312:192.168.202.110@tcp:250:kworker.0
00000100:00100000:3.0:1775766628.362926:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744380 x1862022114541568/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766628.362936:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766628.362938:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775766628.362939:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766628.362941:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4005
00000100:00080000:3.0:1775766628.362943:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766628.362946:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766628.362949:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766628.362956:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746a00 x1862022114541696/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766628.362962:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766628.362964:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775766628.362965:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766628.362968:0:8:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4005
00000100:00080000:3.0:1775766628.362981:0:8:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766628.362984:0:8:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766628.362988:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766628.362995:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746300 x1862022114541824/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:2.0:1775766628.363003:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775766628.363005:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766628.363009:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775766628.363010:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766628.363014:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4005
00000100:00080000:3.0:1775766628.363016:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766628.363019:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766628.363022:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766628.363029:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745880 x1862022114541952/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766628.363034:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766628.363037:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775766628.363038:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766628.363041:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4005
00000100:00080000:3.0:1775766628.363044:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766628.363046:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766628.363049:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766628.363056:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766628/real 1775766628]  req@ffff9f1485747b80 x1862022114541312/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766653 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766628.363059:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747480 x1862022114542080/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766628.363088:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766628.363092:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775766628.363093:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766628.363097:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4005
00000100:00080000:3.0:1775766628.363100:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766628.363104:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775766628.363105:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747b80 x1862022114541312/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766653 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:3.0:1775766628.363107:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766628.363117:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744a80 x1862022114542208/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766628.363128:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766628.363136:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766628.363142:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114541312:192.168.202.110@tcp:250:kworker.0
00000100:00100000:2.0:1775766628.363150:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114541440:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775766628.363171:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766628.363184:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766628/real 1775766628]  req@ffff9f1485744e00 x1862022114541440/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766653 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766628.363198:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744e00 x1862022114541440/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766653 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766628.363207:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766628.363211:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766628.363215:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114541440:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766628.363241:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114541568:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775766628.363261:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766628.363270:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766628/real 1775766628]  req@ffff9f1485744380 x1862022114541568/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766653 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766628.363283:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744380 x1862022114541568/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766653 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766628.363289:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766628.363291:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766628.363293:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114541568:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766628.363297:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114541696:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775766628.363308:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766628.363313:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766628/real 1775766628]  req@ffff9f1485746a00 x1862022114541696/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766653 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766628.363321:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746a00 x1862022114541696/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766653 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766628.363326:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766628.363328:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766628.363330:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114541696:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775766628.363334:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114541824:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775766628.363343:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766628.363348:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766628/real 1775766628]  req@ffff9f1485746300 x1862022114541824/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766653 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766628.363355:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746300 x1862022114541824/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766653 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766628.363364:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766628.363366:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766628.363368:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114541824:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775766628.363371:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114541952:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775766628.363381:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766628.363386:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766628/real 1775766628]  req@ffff9f1485745880 x1862022114541952/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766653 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766628.363392:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745880 x1862022114541952/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766653 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766628.363397:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766628.363399:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766628.363401:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114541952:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766628.363405:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114542080:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775766628.363414:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766628.363419:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766628/real 1775766628]  req@ffff9f1485747480 x1862022114542080/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766653 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766628.363426:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747480 x1862022114542080/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766653 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766628.363431:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766628.363432:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766628.363435:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114542080:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766628.363438:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114542208:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775766628.363447:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766628.363453:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766628/real 1775766628]  req@ffff9f1485744a80 x1862022114542208/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766653 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766628.363460:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744a80 x1862022114542208/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766653 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766628.363465:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766628.363466:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766628.363468:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114542208:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775766628.618788:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766629.386670:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766629.642696:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766630.410652:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766630.666749:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766631.434632:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775766631.690784:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766632.458708:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775766632.714826:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766633.482615:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775766633.482625:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766633.482632:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775766633.482636:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766633.482647:0:8:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4010
00000100:00080000:3.0:1775766633.482658:0:8:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766633.482664:0:8:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775766633.482714:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746680 x1862022114542336/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766633.482757:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766633.482760:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775766633.482762:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766633.482766:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4010
00000100:00080000:3.0:1775766633.482770:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766633.482774:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766633.482784:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766633.482792:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114542336:192.168.202.110@tcp:250:kworker.0
00000100:00100000:3.0:1775766633.482798:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744700 x1862022114542464/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766633.482808:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766633.482812:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775766633.482813:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766633.482817:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4010
00000100:00080000:3.0:1775766633.482819:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766633.482822:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766633.482826:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766633.482835:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745500 x1862022114542592/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766633.482841:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766633.482844:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775766633.482846:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766633.482849:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4010
00000100:00080000:3.0:1775766633.482852:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766633.482854:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766633.482858:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:2.0:1775766633.482863:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766633.482867:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744000 x1862022114542720/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766633.482879:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766633.482883:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775766633.482885:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766633.482890:0:8:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4010
00000100:00080000:3.0:1775766633.482911:0:8:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766633.482916:0:8:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766633.482923:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775766633.482931:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766633/real 1775766633]  req@ffff9f1485746680 x1862022114542336/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766658 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766633.482940:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747800 x1862022114542848/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766633.482962:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766633.482965:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775766633.482967:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:2.0:1775766633.482970:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746680 x1862022114542336/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766658 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766633.482974:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4010
00000100:00080000:3.0:1775766633.482981:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766633.483006:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766633.483015:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00080000:2.0:1775766633.483018:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766633.483024:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766633.483028:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114542336:192.168.202.110@tcp:250:kworker.0
00000100:00100000:3.0:1775766633.483032:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745c00 x1862022114542976/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766633.483040:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766633.483044:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775766633.483045:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766633.483048:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4010
00000100:00080000:3.0:1775766633.483051:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:2.0:1775766633.483056:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114542464:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775766633.483057:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766633.483064:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766633.483074:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745180 x1862022114543104/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:2.0:1775766633.483089:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775766633.483089:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766633.483095:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775766633.483097:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766633.483102:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4010
00000100:00080000:3.0:1775766633.483106:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:2.0:1775766633.483108:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766633/real 1775766633]  req@ffff9f1485744700 x1862022114542464/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766658 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766633.483112:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766633.483119:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775766633.483132:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744700 x1862022114542464/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766658 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766633.483135:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745f80 x1862022114543232/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766633.483182:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766633.483189:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766633.483195:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114542464:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766633.483204:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114542592:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775766633.483231:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766633.483244:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766633/real 1775766633]  req@ffff9f1485745500 x1862022114542592/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766658 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766633.483254:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745500 x1862022114542592/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766658 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766633.483280:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766633.483283:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766633.483295:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114542592:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766633.483302:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114542720:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775766633.483323:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766633.483333:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766633/real 1775766633]  req@ffff9f1485744000 x1862022114542720/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766658 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766633.483355:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744000 x1862022114542720/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766658 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766633.483363:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766633.483366:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766633.483369:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114542720:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775766633.483374:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114542848:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775766633.483399:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766633.483430:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766633/real 1775766633]  req@ffff9f1485747800 x1862022114542848/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766658 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766633.483444:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747800 x1862022114542848/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766658 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766633.483454:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766633.483457:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766633.483461:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114542848:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775766633.483468:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114542976:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775766633.483487:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766633.483496:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766633/real 1775766633]  req@ffff9f1485745c00 x1862022114542976/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766658 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766633.483506:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745c00 x1862022114542976/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766658 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766633.483512:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766633.483514:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766633.483516:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114542976:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766633.483531:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114543104:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775766633.483543:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766633.483550:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766633/real 1775766633]  req@ffff9f1485745180 x1862022114543104/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766658 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766633.483559:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745180 x1862022114543104/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766658 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766633.483565:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766633.483566:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766633.483588:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114543104:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766633.483592:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114543232:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775766633.483606:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766633.483613:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766633/real 1775766633]  req@ffff9f1485745f80 x1862022114543232/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766658 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766633.483622:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745f80 x1862022114543232/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766658 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766633.483633:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766633.483635:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766633.483637:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114543232:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775766633.738778:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766634.506651:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766634.762754:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766635.530634:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766635.786751:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766636.554644:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766636.810730:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766637.578670:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766637.834775:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766638.602658:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775766638.602670:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766638.602681:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775766638.602687:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766638.602700:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4016
00000100:00080000:3.0:1775766638.602715:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766638.602722:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775766638.602768:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747100 x1862022114543360/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766638.602796:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766638.602799:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775766638.602800:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766638.602804:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4016
00000100:00080000:3.0:1775766638.602806:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766638.602810:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766638.602820:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766638.602833:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114543360:192.168.202.110@tcp:250:kworker.0
00000100:00100000:3.0:1775766638.602835:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745f80 x1862022114543488/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766638.602855:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766638.602857:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775766638.602858:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766638.602862:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4016
00000100:00080000:3.0:1775766638.602864:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766638.602867:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766638.602870:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766638.602878:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745180 x1862022114543616/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766638.602884:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766638.602887:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775766638.602888:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766638.602891:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4016
00000100:00080000:3.0:1775766638.602906:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766638.602910:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766638.602913:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766638.602923:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745c00 x1862022114543744/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:2.0:1775766638.602925:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775766638.602938:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766638.602942:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775766638.602944:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766638.602950:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4016
00000100:00080000:3.0:1775766638.602954:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766638.602959:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766638.602962:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775766638.602977:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766638/real 1775766638]  req@ffff9f1485747100 x1862022114543360/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766663 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766638.602979:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747800 x1862022114543872/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766638.603049:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766638.603051:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775766638.603053:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766638.603061:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4016
00000100:00100000:2.0:1775766638.603062:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747100 x1862022114543360/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766663 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766638.603065:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766638.603078:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766638.603092:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00080000:2.0:1775766638.603093:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766638.603100:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766638.603107:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744000 x1862022114544000/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775766638.603109:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114543360:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775766638.603132:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766638.603135:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775766638.603136:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766638.603140:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4016
00000100:00080000:3.0:1775766638.603144:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766638.603153:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775766638.603154:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114543488:192.168.202.110@tcp:38:kworker.0
00000008:00100000:3.0:1775766638.603169:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766638.603184:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745500 x1862022114544128/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:2.0:1775766638.603195:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775766638.603211:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:2.0:1775766638.603215:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766638/real 1775766638]  req@ffff9f1485745f80 x1862022114543488/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766663 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766638.603216:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775766638.603218:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766638.603234:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4016
00000100:00080000:3.0:1775766638.603245:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766638.603255:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766638.603260:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775766638.603261:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745f80 x1862022114543488/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766663 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766638.603278:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00100000:3.0:1775766638.603282:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744700 x1862022114544256/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766638.603286:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766638.603289:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114543488:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766638.603299:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114543616:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775766638.603335:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766638.603350:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766638/real 1775766638]  req@ffff9f1485745180 x1862022114543616/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766663 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766638.603359:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745180 x1862022114543616/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766663 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766638.603364:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766638.603366:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766638.603368:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114543616:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766638.603373:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114543744:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775766638.603384:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766638.603389:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766638/real 1775766638]  req@ffff9f1485745c00 x1862022114543744/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766663 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766638.603395:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745c00 x1862022114543744/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766663 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766638.603402:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766638.603404:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766638.603406:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114543744:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775766638.603410:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114543872:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775766638.603419:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766638.603424:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766638/real 1775766638]  req@ffff9f1485747800 x1862022114543872/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766663 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766638.603430:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747800 x1862022114543872/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766663 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766638.603435:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766638.603437:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766638.603439:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114543872:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775766638.603444:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114544000:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775766638.603453:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766638.603458:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766638/real 1775766638]  req@ffff9f1485744000 x1862022114544000/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766663 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766638.603465:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744000 x1862022114544000/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766663 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766638.603470:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766638.603472:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766638.603475:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114544000:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766638.603487:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114544128:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775766638.603498:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766638.603503:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766638/real 1775766638]  req@ffff9f1485745500 x1862022114544128/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766663 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766638.603511:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745500 x1862022114544128/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766663 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766638.603516:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766638.603518:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766638.603519:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114544128:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766638.603523:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114544256:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775766638.603533:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766638.603538:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766638/real 1775766638]  req@ffff9f1485744700 x1862022114544256/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766663 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766638.603546:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744700 x1862022114544256/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766663 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766638.603551:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766638.603573:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766638.603575:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114544256:192.168.202.110@tcp:8:kworker.0
00010000:00010000:3.0:1775766638.858732:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766639.626726:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775766639.882697:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766640.650687:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766640.906729:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766641.674720:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766641.930819:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766642.698709:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766642.954726:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1775766643.722677:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:2.0:1775766643.722682:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775766643.722693:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775766643.722699:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766643.722711:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4021
00000100:00080000:3.0:1775766643.722724:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766643.722731:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775766643.722777:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746680 x1862022114544384/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766643.722805:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766643.722808:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775766643.722810:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766643.722813:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4021
00000100:00080000:3.0:1775766643.722816:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766643.722819:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766643.722829:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766643.722840:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114544384:192.168.202.110@tcp:250:kworker.0
00000100:00100000:3.0:1775766643.722841:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744a80 x1862022114544512/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766643.722853:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766643.722855:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775766643.722856:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766643.722859:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4021
00000100:00080000:3.0:1775766643.722861:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766643.722863:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766643.722866:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766643.722874:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747480 x1862022114544640/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766643.722881:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766643.722883:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775766643.722884:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766643.722887:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4021
00000100:00080000:3.0:1775766643.722904:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766643.722907:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766643.722911:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766643.722918:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745880 x1862022114544768/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:2.0:1775766643.722923:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775766643.722936:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766643.722938:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775766643.722940:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766643.722944:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4021
00000100:00080000:3.0:1775766643.722947:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766643.722949:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766643.722952:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766643.722960:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746300 x1862022114544896/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766643.722966:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766643.722969:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00100000:2.0:1775766643.722972:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766643/real 1775766643]  req@ffff9f1485746680 x1862022114544384/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766668 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766643.722972:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766643.722977:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4021
00000100:00080000:3.0:1775766643.722981:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766643.722987:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766643.722991:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766643.722998:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746680 x1862022114544384/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766668 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766643.723003:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746a00 x1862022114545024/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766643.723014:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766643.723018:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766643.723020:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:3.0:1775766643.723026:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00100000:2.0:1775766643.723027:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114544384:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775766643.723030:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766643.723036:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4021
00000100:00080000:3.0:1775766643.723041:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766643.723046:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775766643.723051:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114544512:192.168.202.110@tcp:38:kworker.0
00000008:00100000:3.0:1775766643.723052:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766643.723071:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744380 x1862022114545152/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:2.0:1775766643.723079:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775766643.723089:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:2.0:1775766643.723092:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766643/real 1775766643]  req@ffff9f1485744a80 x1862022114544512/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766668 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766643.723095:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775766643.723099:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766643.723108:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4021
00000100:00100000:2.0:1775766643.723111:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744a80 x1862022114544512/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766668 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766643.723114:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766643.723122:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766643.723123:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00080000:2.0:1775766643.723130:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000008:00100000:3.0:1775766643.723132:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775766643.723133:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114544512:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766643.723142:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114544640:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766643.723151:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744e00 x1862022114545280/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:2.0:1775766643.723159:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766643.723171:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766643/real 1775766643]  req@ffff9f1485747480 x1862022114544640/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766668 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766643.723180:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747480 x1862022114544640/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766668 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766643.723187:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766643.723190:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766643.723193:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114544640:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766643.723197:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114544768:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775766643.723208:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766643.723213:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766643/real 1775766643]  req@ffff9f1485745880 x1862022114544768/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766668 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766643.723220:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745880 x1862022114544768/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766668 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766643.723226:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766643.723227:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766643.723229:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114544768:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775766643.723233:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114544896:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775766643.723243:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766643.723249:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766643/real 1775766643]  req@ffff9f1485746300 x1862022114544896/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766668 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766643.723256:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746300 x1862022114544896/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766668 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766643.723261:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766643.723262:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766643.723264:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114544896:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775766643.723267:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114545024:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775766643.723277:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766643.723283:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766643/real 1775766643]  req@ffff9f1485746a00 x1862022114545024/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766668 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766643.723291:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746a00 x1862022114545024/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766668 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766643.723296:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766643.723297:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766643.723299:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114545024:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766643.723310:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114545152:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775766643.723320:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766643.723326:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766643/real 1775766643]  req@ffff9f1485744380 x1862022114545152/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766668 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766643.723333:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744380 x1862022114545152/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766668 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766643.723342:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766643.723344:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766643.723346:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114545152:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766643.723349:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114545280:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775766643.723359:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766643.723364:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766643/real 1775766643]  req@ffff9f1485744e00 x1862022114545280/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766668 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766643.723370:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744e00 x1862022114545280/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766668 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766643.723376:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766643.723377:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766643.723379:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114545280:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775766643.978788:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766644.746724:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766645.002712:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766645.770721:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775766646.026751:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766646.794632:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775766647.050710:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766647.818643:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766648.074696:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766648.842661:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775766648.842676:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766648.842685:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775766648.842690:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766648.842703:0:8:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4026
00000100:00080000:3.0:1775766648.842717:0:8:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766648.842726:0:8:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775766648.842782:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747b80 x1862022114545408/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766648.842840:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766648.842844:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775766648.842845:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766648.842849:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4026
00000100:00080000:3.0:1775766648.842854:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766648.842858:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766648.842868:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766648.842874:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114545408:192.168.202.110@tcp:250:kworker.0
00000100:00100000:3.0:1775766648.842885:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744e00 x1862022114545536/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766648.842895:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766648.842915:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775766648.842917:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766648.842920:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4026
00000100:00080000:3.0:1775766648.842923:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766648.842926:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766648.842930:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766648.842940:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744380 x1862022114545664/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766648.842946:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766648.842949:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775766648.842951:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766648.842954:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4026
00000100:00080000:3.0:1775766648.842957:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766648.842959:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766648.842964:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:2.0:1775766648.842966:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766648.842976:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746a00 x1862022114545792/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766648.842984:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766648.842987:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775766648.842989:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766648.842991:0:8:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4026
00000100:00080000:3.0:1775766648.842994:0:8:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766648.842997:0:8:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766648.843000:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766648.843012:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746300 x1862022114545920/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775766648.843014:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766648/real 1775766648]  req@ffff9f1485747b80 x1862022114545408/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766673 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766648.843035:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766648.843040:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775766648.843041:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766648.843045:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4026
00000100:00080000:3.0:1775766648.843048:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:2.0:1775766648.843052:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747b80 x1862022114545408/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766673 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766648.843056:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766648.843063:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00080000:2.0:1775766648.843071:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00100000:3.0:1775766648.843075:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745880 x1862022114546048/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766648.843077:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766648.843086:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114545408:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775766648.843093:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766648.843096:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00100000:2.0:1775766648.843112:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114545536:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775766648.843130:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775766648.843135:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766648.843142:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4026
00000100:00100000:2.0:1775766648.843143:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766648/real 1775766648]  req@ffff9f1485744e00 x1862022114545536/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766673 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766648.843148:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766648.843157:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766648.843163:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766648.843165:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744e00 x1862022114545536/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766673 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766648.843176:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747480 x1862022114546176/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766648.843186:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766648.843186:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766648.843201:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775766648.843204:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00080000:3.0:1775766648.843205:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:2.0:1775766648.843211:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114545536:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775766648.843213:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4026
00000100:00080000:3.0:1775766648.843221:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766648.843224:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775766648.843229:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114545664:192.168.202.110@tcp:38:kworker.0
00000008:00100000:3.0:1775766648.843229:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766648.843241:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744a80 x1862022114546304/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:2.0:1775766648.843285:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766648.843315:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766648/real 1775766648]  req@ffff9f1485744380 x1862022114545664/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766673 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766648.843330:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744380 x1862022114545664/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766673 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766648.843343:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766648.843346:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766648.843349:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114545664:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766648.843355:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114545792:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775766648.843391:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766648.843399:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766648/real 1775766648]  req@ffff9f1485746a00 x1862022114545792/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766673 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766648.843408:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746a00 x1862022114545792/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766673 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766648.843413:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766648.843416:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766648.843418:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114545792:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775766648.843422:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114545920:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775766648.843432:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766648.843438:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766648/real 1775766648]  req@ffff9f1485746300 x1862022114545920/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766673 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766648.843445:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746300 x1862022114545920/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766673 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766648.843451:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766648.843453:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766648.843456:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114545920:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775766648.843461:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114546048:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775766648.843481:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766648.843505:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766648/real 1775766648]  req@ffff9f1485745880 x1862022114546048/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766673 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766648.843518:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745880 x1862022114546048/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766673 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766648.843528:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766648.843531:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766648.843535:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114546048:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766648.843611:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114546176:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775766648.843650:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766648.843660:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766648/real 1775766648]  req@ffff9f1485747480 x1862022114546176/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766673 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766648.843671:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747480 x1862022114546176/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766673 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766648.843677:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766648.843680:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766648.843682:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114546176:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766648.843687:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114546304:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775766648.843699:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766648.843706:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766648/real 1775766648]  req@ffff9f1485744a80 x1862022114546304/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766673 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766648.843715:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744a80 x1862022114546304/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766673 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766648.843722:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766648.843724:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766648.843726:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114546304:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775766649.098823:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766649.866755:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766650.122846:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766650.890633:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766651.146768:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766651.914650:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766652.170766:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766652.938622:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775766653.194737:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1775766653.962666:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766653.962676:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00100000:2.0:1775766653.962677:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775766653.962680:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766653.962690:0:8:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4031
00000100:00080000:3.0:1775766653.962703:0:8:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766653.962710:0:8:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775766653.962755:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746680 x1862022114546432/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766653.962784:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766653.962787:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775766653.962788:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766653.962792:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4031
00000100:00080000:3.0:1775766653.962794:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766653.962797:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766653.962807:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766653.962816:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114546432:192.168.202.110@tcp:250:kworker.0
00000100:00100000:3.0:1775766653.962817:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744700 x1862022114546560/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766653.962829:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766653.962832:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775766653.962833:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766653.962835:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4031
00000100:00080000:3.0:1775766653.962837:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766653.962839:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766653.962842:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766653.962850:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745500 x1862022114546688/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766653.962856:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766653.962858:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775766653.962859:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766653.962861:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4031
00000100:00080000:3.0:1775766653.962872:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766653.962876:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766653.962879:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:2.0:1775766653.962889:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766653.962889:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744000 x1862022114546816/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766653.962897:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766653.962900:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775766653.962902:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766653.962905:0:8:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4031
00000100:00080000:3.0:1775766653.962907:0:8:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766653.962910:0:8:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766653.962913:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766653.962920:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747800 x1862022114546944/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766653.962925:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766653.962928:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775766653.962930:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766653.962933:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4031
00000100:00080000:3.0:1775766653.962936:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:2.0:1775766653.962939:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766653/real 1775766653]  req@ffff9f1485746680 x1862022114546432/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766678 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766653.962943:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766653.962949:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766653.962961:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745c00 x1862022114547072/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775766653.962966:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746680 x1862022114546432/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766678 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766653.962988:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766653.962992:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775766653.962994:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766653.962996:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766653.962998:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4031
00000100:00080000:3.0:1775766653.963003:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766653.963007:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:3.0:1775766653.963008:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766653.963014:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766653.963019:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114546432:192.168.202.110@tcp:250:kworker.0
00000100:00100000:3.0:1775766653.963025:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745180 x1862022114547200/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766653.963032:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766653.963035:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775766653.963036:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766653.963039:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4031
00000100:00080000:3.0:1775766653.963041:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766653.963043:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766653.963046:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775766653.963050:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114546560:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766653.963054:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745f80 x1862022114547328/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:2.0:1775766653.963077:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766653.963089:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766653/real 1775766653]  req@ffff9f1485744700 x1862022114546560/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766678 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766653.963099:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744700 x1862022114546560/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766678 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766653.963104:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766653.963107:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766653.963109:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114546560:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766653.963113:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114546688:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775766653.963124:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766653.963129:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766653/real 1775766653]  req@ffff9f1485745500 x1862022114546688/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766678 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766653.963137:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745500 x1862022114546688/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766678 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766653.963142:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766653.963144:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766653.963145:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114546688:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766653.963150:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114546816:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775766653.963161:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766653.963166:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766653/real 1775766653]  req@ffff9f1485744000 x1862022114546816/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766678 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766653.963174:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744000 x1862022114546816/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766678 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766653.963179:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766653.963181:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766653.963183:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114546816:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775766653.963186:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114546944:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775766653.963195:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766653.963200:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766653/real 1775766653]  req@ffff9f1485747800 x1862022114546944/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766678 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766653.963207:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747800 x1862022114546944/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766678 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766653.963212:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766653.963214:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766653.963215:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114546944:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775766653.963219:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114547072:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775766653.963228:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766653.963233:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766653/real 1775766653]  req@ffff9f1485745c00 x1862022114547072/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766678 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766653.963240:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745c00 x1862022114547072/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766678 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766653.963249:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766653.963251:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766653.963253:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114547072:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766653.963256:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114547200:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775766653.963265:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766653.963270:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766653/real 1775766653]  req@ffff9f1485745180 x1862022114547200/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766678 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766653.963276:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745180 x1862022114547200/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766678 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766653.963281:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766653.963283:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766653.963285:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114547200:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766653.963298:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114547328:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775766653.963308:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766653.963313:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766653/real 1775766653]  req@ffff9f1485745f80 x1862022114547328/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766678 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766653.963320:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745f80 x1862022114547328/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766678 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766653.963325:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766653.963327:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766653.963329:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114547328:192.168.202.110@tcp:8:kworker.0
00010000:00010000:1.0:1775766654.218807:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766654.986736:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766655.242737:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766656.010676:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766656.266753:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766657.034737:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766657.290741:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766658.058629:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766658.314723:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766659.082676:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775766659.082696:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766659.082716:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775766659.082725:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766659.082736:0:8:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4036
00000100:00080000:3.0:1775766659.082750:0:8:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766659.082757:0:8:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775766659.082797:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747100 x1862022114547456/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766659.082831:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766659.082835:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775766659.082837:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766659.082843:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4036
00000100:00080000:3.0:1775766659.082849:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766659.082855:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766659.082866:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766659.082869:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114547456:192.168.202.110@tcp:250:kworker.0
00000100:00100000:3.0:1775766659.082892:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745f80 x1862022114547584/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766659.082907:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766659.082911:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775766659.082913:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766659.082918:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4036
00000100:00080000:3.0:1775766659.082933:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766659.082939:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766659.082945:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000800:00000100:2.0:1775766659.082951:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766659.082964:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745180 x1862022114547712/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766659.082978:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766659.082983:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775766659.082985:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766659.082992:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4036
00000100:00080000:3.0:1775766659.083000:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00100000:2.0:1775766659.083008:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766659/real 1775766659]  req@ffff9f1485747100 x1862022114547456/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766684 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766659.083009:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766659.083020:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775766659.083042:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747100 x1862022114547456/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766684 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766659.083047:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745c00 x1862022114547840/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766659.083071:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766659.083072:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766659.083082:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:3.0:1775766659.083083:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775766659.083089:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:2.0:1775766659.083092:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114547456:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775766659.083101:0:8:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4036
00000100:00080000:3.0:1775766659.083107:0:8:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766659.083112:0:8:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766659.083120:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775766659.083126:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114547584:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766659.083140:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747800 x1862022114547968/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:2.0:1775766659.083151:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775766659.083162:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766659.083168:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00100000:2.0:1775766659.083171:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766659/real 1775766659]  req@ffff9f1485745f80 x1862022114547584/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766684 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766659.083172:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:2.0:1775766659.083199:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745f80 x1862022114547584/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766684 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766659.083200:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4036
00000100:00080000:3.0:1775766659.083215:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766659.083216:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766659.083225:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00080000:3.0:1775766659.083231:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775766659.083232:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114547584:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766659.083249:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114547712:192.168.202.110@tcp:38:kworker.0
00000008:00100000:3.0:1775766659.083250:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000800:00000100:2.0:1775766659.083282:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766659.083284:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744000 x1862022114548096/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775766659.083291:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766659/real 1775766659]  req@ffff9f1485745180 x1862022114547712/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766684 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766659.083316:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766659.083329:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775766659.083333:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:2.0:1775766659.083339:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745180 x1862022114547712/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766684 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766659.083340:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4036
00000100:00080000:3.0:1775766659.083350:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766659.083359:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00080000:2.0:1775766659.083363:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000008:00100000:3.0:1775766659.083366:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00080000:2.0:1775766659.083370:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766659.083380:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114547712:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766659.083389:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745500 x1862022114548224/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775766659.083394:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114547840:192.168.202.110@tcp:8:kworker.0
00000100:00080000:3.0:1775766659.083417:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766659.083421:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775766659.083424:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766659.083429:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4036
00000100:00080000:3.0:1775766659.083448:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766659.083454:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766659.083460:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:2.0:1775766659.083468:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766659.083481:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744700 x1862022114548352/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775766659.083485:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766659/real 1775766659]  req@ffff9f1485745c00 x1862022114547840/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766684 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766659.083545:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745c00 x1862022114547840/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766684 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766659.083578:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766659.083581:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766659.083584:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114547840:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775766659.083617:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114547968:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775766659.083650:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766659.083663:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766659/real 1775766659]  req@ffff9f1485747800 x1862022114547968/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766684 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766659.083678:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747800 x1862022114547968/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766684 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766659.083689:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766659.083691:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766659.083693:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114547968:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775766659.083699:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114548096:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775766659.083720:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766659.083730:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766659/real 1775766659]  req@ffff9f1485744000 x1862022114548096/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766684 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766659.083739:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744000 x1862022114548096/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766684 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766659.083746:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766659.083750:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766659.083754:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114548096:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766659.083760:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114548224:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775766659.083782:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766659.083802:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766659/real 1775766659]  req@ffff9f1485745500 x1862022114548224/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766684 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766659.083820:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745500 x1862022114548224/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766684 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766659.083828:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766659.083834:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766659.083839:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114548224:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766659.083846:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114548352:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775766659.083861:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766659.083870:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766659/real 1775766659]  req@ffff9f1485744700 x1862022114548352/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766684 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766659.083878:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744700 x1862022114548352/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766684 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766659.083884:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766659.083886:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766659.083889:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114548352:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775766659.338808:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766660.106697:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775766660.362726:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766661.130664:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775766661.386791:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766662.154624:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766662.410704:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766663.178667:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766663.434702:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1775766664.202631:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766664.202642:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1775766664.202647:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00100000:2.0:1775766664.202654:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:0.0:1775766664.202659:0:8:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4041
00000100:00080000:0.0:1775766664.202672:0:8:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766664.202691:0:8:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775766664.202736:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d1180 x1862022114548480/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775766664.202763:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766664.202767:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775766664.202768:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766664.202771:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4041
00000100:00080000:0.0:1775766664.202774:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766664.202777:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775766664.202787:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766664.202795:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114548480:192.168.202.110@tcp:250:kworker.0
00000100:00100000:0.0:1775766664.202799:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3800 x1862022114548608/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775766664.202808:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766664.202811:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:0.0:1775766664.202812:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766664.202814:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4041
00000100:00080000:0.0:1775766664.202816:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766664.202818:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775766664.202823:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775766664.202839:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d1500 x1862022114548736/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775766664.202847:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766664.202849:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775766664.202850:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766664.202853:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4041
00000100:00080000:0.0:1775766664.202856:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766664.202859:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775766664.202863:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775766664.202870:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0000 x1862022114548864/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775766664.202877:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766664.202880:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1775766664.202881:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766664.202883:0:8:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4041
00000100:00080000:0.0:1775766664.202885:0:8:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766664.202887:0:8:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775766664.202890:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775766664.202899:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3480 x1862022114548992/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775766664.202904:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766664.202907:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775766664.202909:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:2.0:1775766664.202911:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114548608:192.168.202.110@tcp:38:kworker.0
00000100:00080000:0.0:1775766664.202915:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4041
00000100:00080000:0.0:1775766664.202919:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766664.202923:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775766664.202927:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775766664.202939:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d1f80 x1862022114549120/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775766664.202943:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114548736:192.168.202.110@tcp:38:kworker.0
00000100:00080000:0.0:1775766664.202955:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766664.202957:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:0.0:1775766664.202958:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766664.202962:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4041
00000100:00080000:0.0:1775766664.202966:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766664.202971:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775766664.202971:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114548864:192.168.202.110@tcp:8:kworker.0
00000008:00100000:0.0:1775766664.202980:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775766664.202993:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0380 x1862022114549248/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775766664.202998:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114548992:192.168.202.110@tcp:8:kworker.0
00000100:00080000:0.0:1775766664.203008:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766664.203010:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775766664.203012:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766664.203016:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4041
00000100:00080000:0.0:1775766664.203020:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766664.203024:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775766664.203024:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114549120:192.168.202.110@tcp:38:kworker.0
00000008:00100000:0.0:1775766664.203032:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775766664.203044:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0e00 x1862022114549376/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775766664.203050:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114549248:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766664.203074:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114549376:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775766664.458745:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766665.226664:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766665.482820:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766666.250686:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766666.506773:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766667.274705:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000400:00000100:2.0:1775766667.274995:0:2391:0:(lib-socket.c:454:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.202.110:988
00000400:00000100:2.0:1775766667.275051:0:2391:0:(acceptor.c:111:lnet_connect_console_error()) Connection to 192.168.202.110@tcp at host 192.168.202.110 was unreachable: the network or that node may be down, or Lustre may be misconfigured.
00000800:00000100:2.0:1775766667.275075:0:2391:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:2.0:1775766667.275135:0:2391:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:2.0:1775766667.275146:0:2391:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:2.0:1775766667.275159:0:2391:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:2.0:1775766667.275171:0:2391:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:2.0:1775766667.275181:0:2391:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:2.0:1775766667.275192:0:2391:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000100:00100000:0.0:1775766667.275203:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766664/real 1775766667]  req@ffff9f14a04d3800 x1862022114548608/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766689 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000800:00000100:2.0:1775766667.275205:0:2391:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000100:00100000:0.0:1775766667.275248:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3800 x1862022114548608/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766689 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766667.275265:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766667.275271:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766667.275276:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114548608:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766667.275286:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766664/real 1775766667]  req@ffff9f14a04d1500 x1862022114548736/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766689 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766667.275300:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1500 x1862022114548736/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766689 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766667.275309:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766667.275313:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766667.275317:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114548736:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766667.275324:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766664/real 1775766667]  req@ffff9f14a04d0000 x1862022114548864/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766689 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766667.275337:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0000 x1862022114548864/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766689 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766667.275390:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766667.275397:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766667.275401:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114548864:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775766667.275410:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766664/real 1775766667]  req@ffff9f14a04d3480 x1862022114548992/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766689 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766667.275424:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3480 x1862022114548992/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766689 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766667.275433:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766667.275438:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766667.275441:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114548992:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775766667.275450:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766664/real 1775766667]  req@ffff9f14a04d1180 x1862022114548480/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766689 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766667.275464:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1180 x1862022114548480/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766689 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766667.275473:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766667.275478:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766667.275482:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114548480:192.168.202.110@tcp:250:kworker.0
00000100:00100000:0.0:1775766667.275531:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766664/real 1775766667]  req@ffff9f14a04d0e00 x1862022114549376/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766689 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766667.275549:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0e00 x1862022114549376/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766689 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766667.275592:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766667.275600:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766667.275606:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114549376:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775766667.275617:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766664/real 1775766667]  req@ffff9f14a04d1f80 x1862022114549120/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766689 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766667.275636:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1f80 x1862022114549120/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766689 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766667.275647:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766667.275652:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766667.275657:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114549120:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766667.275666:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766664/real 1775766667]  req@ffff9f14a04d0380 x1862022114549248/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766689 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766667.275683:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0380 x1862022114549248/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766689 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766667.275693:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766667.275698:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766667.275702:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114549248:192.168.202.110@tcp:38:kworker.0
00010000:00010000:3.0:1775766667.530728:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766668.298655:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775766668.554819:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766669.322683:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:0.0:1775766669.322725:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766669.322737:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1775766669.322743:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766669.322756:0:8:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4046
00000100:00080000:0.0:1775766669.322771:0:8:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766669.322780:0:8:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775766669.322828:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0380 x1862022114549504/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775766669.322871:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766669.322875:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775766669.322877:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766669.322882:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4046
00000100:00080000:0.0:1775766669.322887:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766669.322892:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775766669.322905:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766669.322920:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114549504:192.168.202.110@tcp:250:kworker.0
00000100:00100000:0.0:1775766669.322922:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d1f80 x1862022114549632/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775766669.322938:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766669.322941:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:0.0:1775766669.322943:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766669.322948:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4046
00000100:00080000:0.0:1775766669.322952:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766669.322956:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775766669.322963:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775766669.322976:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0e00 x1862022114549760/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775766669.322988:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766669.322992:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775766669.322993:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766669.322998:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4046
00000100:00080000:0.0:1775766669.323002:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766669.323008:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000800:00000100:2.0:1775766669.323012:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000008:00100000:0.0:1775766669.323017:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775766669.323031:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d1180 x1862022114549888/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775766669.323042:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766669.323046:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1775766669.323048:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766669.323052:0:8:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4046
00000100:00080000:0.0:1775766669.323057:0:8:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766669.323062:0:8:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775766669.323066:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766669/real 1775766669]  req@ffff9f14a04d0380 x1862022114549504/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766694 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:0.0:1775766669.323070:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775766669.323088:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3480 x1862022114550016/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775766669.323097:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0380 x1862022114549504/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766694 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766669.323105:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766669.323116:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775766669.323119:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766669.323121:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766669.323126:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4046
00000100:00080000:2.0:1775766669.323129:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766669.323133:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114549504:192.168.202.110@tcp:250:kworker.0
00000100:00080000:0.0:1775766669.323155:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766669.323162:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775766669.323168:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114549632:192.168.202.110@tcp:38:kworker.0
00000008:00100000:0.0:1775766669.323169:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775766669.323199:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0000 x1862022114550144/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:2.0:1775766669.323206:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:0.0:1775766669.323220:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766669.323224:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:0.0:1775766669.323227:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:2.0:1775766669.323227:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766669/real 1775766669]  req@ffff9f14a04d1f80 x1862022114549632/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766694 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766669.323237:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4046
00000100:00080000:0.0:1775766669.323246:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766669.323252:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775766669.323255:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1f80 x1862022114549632/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766694 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:0.0:1775766669.323262:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00080000:2.0:1775766669.323267:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766669.323273:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766669.323279:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114549632:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766669.323286:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114549760:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766669.323289:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d1500 x1862022114550272/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775766669.323304:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766669.323308:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775766669.323310:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766669.323315:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4046
00000800:00000100:2.0:1775766669.323321:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:0.0:1775766669.323336:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766669.323344:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775766669.323351:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775766669.323356:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766669/real 1775766669]  req@ffff9f14a04d0e00 x1862022114549760/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766694 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766669.323369:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3800 x1862022114550400/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775766669.323372:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0e00 x1862022114549760/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766694 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766669.323398:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766669.323404:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766669.323407:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114549760:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766669.323415:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114549888:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775766669.323439:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766669.323450:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766669/real 1775766669]  req@ffff9f14a04d1180 x1862022114549888/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766694 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766669.323462:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1180 x1862022114549888/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766694 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766669.323470:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766669.323473:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766669.323477:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114549888:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775766669.323483:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114550016:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775766669.323505:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766669.323517:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766669/real 1775766669]  req@ffff9f14a04d3480 x1862022114550016/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766694 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766669.323536:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3480 x1862022114550016/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766694 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766669.323544:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766669.323547:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766669.323551:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114550016:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775766669.323603:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114550144:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775766669.323628:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766669.323641:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766669/real 1775766669]  req@ffff9f14a04d0000 x1862022114550144/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766694 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766669.323654:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0000 x1862022114550144/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766694 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766669.323663:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766669.323667:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766669.323670:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114550144:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766669.323677:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114550272:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775766669.323699:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766669.323711:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766669/real 1775766669]  req@ffff9f14a04d1500 x1862022114550272/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766694 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766669.323731:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1500 x1862022114550272/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766694 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766669.323746:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766669.323750:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766669.323753:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114550272:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766669.323759:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114550400:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775766669.323785:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766669.323796:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766669/real 1775766669]  req@ffff9f14a04d3800 x1862022114550400/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766694 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766669.323807:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3800 x1862022114550400/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766694 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766669.323815:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766669.323819:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766669.323822:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114550400:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775766669.578806:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766670.346653:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766670.602736:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766671.370683:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766671.626708:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766672.394680:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766672.650776:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766673.418649:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766673.674753:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766674.442643:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775766674.442654:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766674.442660:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775766674.442663:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766674.442672:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4051
00000100:00080000:3.0:1775766674.442682:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766674.442688:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775766674.442723:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746680 x1862022114550528/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766674.442746:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766674.442749:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775766674.442750:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766674.442753:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4051
00000100:00080000:3.0:1775766674.442756:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766674.442759:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766674.442767:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766674.442777:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114550528:192.168.202.110@tcp:250:kworker.0
00000100:00100000:3.0:1775766674.442778:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744a80 x1862022114550656/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766674.442795:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766674.442798:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775766674.442799:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766674.442801:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4051
00000100:00080000:3.0:1775766674.442803:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766674.442805:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766674.442808:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766674.442816:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747480 x1862022114550784/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766674.442823:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766674.442825:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775766674.442826:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766674.442828:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4051
00000100:00080000:3.0:1775766674.442838:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766674.442840:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766674.442843:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766674.442850:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745880 x1862022114550912/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:2.0:1775766674.442854:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775766674.442862:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766674.442864:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775766674.442866:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766674.442870:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4051
00000100:00080000:3.0:1775766674.442873:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766674.442876:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766674.442878:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766674.442885:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746300 x1862022114551040/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766674.442890:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766674.442893:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775766674.442894:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:2.0:1775766674.442898:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766674/real 1775766674]  req@ffff9f1485746680 x1862022114550528/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766699 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766674.442899:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4051
00000100:00080000:3.0:1775766674.442905:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766674.442912:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766674.442916:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766674.442924:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746680 x1862022114550528/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766699 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766674.442925:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746a00 x1862022114551168/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766674.442957:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766674.442959:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766674.442960:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775766674.442964:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766674.442968:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:3.0:1775766674.442971:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4051
00000100:00080000:3.0:1775766674.442974:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:2.0:1775766674.442976:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114550528:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775766674.442979:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766674.442984:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766674.442992:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744380 x1862022114551296/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766674.442998:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766674.443001:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00100000:2.0:1775766674.443003:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114550656:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775766674.443003:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766674.443011:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4051
00000100:00080000:3.0:1775766674.443013:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766674.443015:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766674.443018:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766674.443027:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744e00 x1862022114551424/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:2.0:1775766674.443030:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766674.443043:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766674/real 1775766674]  req@ffff9f1485744a80 x1862022114550656/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766699 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766674.443054:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744a80 x1862022114550656/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766699 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766674.443061:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766674.443063:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766674.443065:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114550656:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766674.443070:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114550784:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775766674.443080:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766674.443085:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766674/real 1775766674]  req@ffff9f1485747480 x1862022114550784/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766699 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766674.443098:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747480 x1862022114550784/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766699 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766674.443103:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766674.443105:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766674.443106:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114550784:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766674.443110:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114550912:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775766674.443120:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766674.443125:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766674/real 1775766674]  req@ffff9f1485745880 x1862022114550912/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766699 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766674.443133:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745880 x1862022114550912/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766699 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766674.443138:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766674.443140:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766674.443141:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114550912:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775766674.443144:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114551040:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775766674.443153:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766674.443158:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766674/real 1775766674]  req@ffff9f1485746300 x1862022114551040/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766699 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766674.443164:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746300 x1862022114551040/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766699 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766674.443173:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766674.443175:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766674.443177:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114551040:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775766674.443180:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114551168:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775766674.443189:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766674.443194:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766674/real 1775766674]  req@ffff9f1485746a00 x1862022114551168/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766699 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766674.443200:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746a00 x1862022114551168/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766699 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766674.443205:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766674.443207:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766674.443209:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114551168:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766674.443213:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114551296:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775766674.443222:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766674.443227:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766674/real 1775766674]  req@ffff9f1485744380 x1862022114551296/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766699 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766674.443233:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744380 x1862022114551296/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766699 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766674.443238:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766674.443240:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766674.443242:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114551296:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766674.443254:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114551424:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775766674.443264:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766674.443293:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766674/real 1775766674]  req@ffff9f1485744e00 x1862022114551424/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766699 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766674.443303:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744e00 x1862022114551424/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766699 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766674.443308:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766674.443310:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766674.443312:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114551424:192.168.202.110@tcp:8:kworker.0
00010000:00010000:3.0:1775766674.698748:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766675.466677:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775766675.722658:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766676.490750:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766676.746734:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766677.514628:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766677.770757:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766678.538640:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766678.794701:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766679.562631:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775766679.562673:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766679.562685:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775766679.562690:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766679.562702:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4057
00000100:00080000:3.0:1775766679.562714:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766679.562724:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775766679.562766:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747b80 x1862022114551552/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766679.562794:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766679.562798:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775766679.562799:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766679.562803:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4057
00000100:00080000:3.0:1775766679.562807:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766679.562811:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766679.562820:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766679.562841:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114551552:192.168.202.110@tcp:250:kworker.0
00000100:00100000:3.0:1775766679.562841:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744e00 x1862022114551680/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766679.562870:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766679.562873:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775766679.562874:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766679.562880:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4057
00000100:00080000:3.0:1775766679.562886:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766679.562891:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766679.562895:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766679.562909:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744380 x1862022114551808/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766679.562916:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766679.562920:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775766679.562922:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766679.562925:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4057
00000100:00080000:3.0:1775766679.562928:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766679.562931:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766679.562935:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:2.0:1775766679.562941:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766679.562951:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746a00 x1862022114551936/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766679.562963:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766679.562966:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775766679.562969:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766679.562972:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4057
00000100:00080000:3.0:1775766679.562975:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766679.562978:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766679.562983:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766679.563000:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746300 x1862022114552064/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775766679.563006:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766679/real 1775766679]  req@ffff9f1485747b80 x1862022114551552/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766704 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766679.563047:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766679.563051:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775766679.563052:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766679.563059:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4057
00000100:00100000:2.0:1775766679.563060:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747b80 x1862022114551552/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766704 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766679.563068:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766679.563078:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766679.563090:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766679.563112:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745880 x1862022114552192/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766679.563119:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766679.563120:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766679.563129:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775766679.563131:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766679.563136:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:3.0:1775766679.563143:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4057
00000100:00100000:2.0:1775766679.563147:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114551552:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775766679.563150:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766679.563161:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766679.563169:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766679.563187:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747480 x1862022114552320/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775766679.563191:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114551680:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775766679.563232:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766679.563236:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775766679.563239:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766679.563246:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4057
00000100:00080000:3.0:1775766679.563251:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766679.563257:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000800:00000100:2.0:1775766679.563258:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000008:00100000:3.0:1775766679.563269:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775766679.563285:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766679/real 1775766679]  req@ffff9f1485744e00 x1862022114551680/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766704 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766679.563286:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744a80 x1862022114552448/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775766679.563372:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744e00 x1862022114551680/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766704 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766679.563380:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766679.563383:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766679.563386:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114551680:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766679.563394:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114551808:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775766679.563411:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766679.563420:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766679/real 1775766679]  req@ffff9f1485744380 x1862022114551808/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766704 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766679.563434:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744380 x1862022114551808/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766704 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766679.563443:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766679.563446:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766679.563450:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114551808:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766679.563458:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114551936:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775766679.563480:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766679.563492:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766679/real 1775766679]  req@ffff9f1485746a00 x1862022114551936/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766704 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766679.563501:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746a00 x1862022114551936/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766704 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766679.563506:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766679.563509:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766679.563511:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114551936:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775766679.563515:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114552064:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775766679.563530:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766679.563536:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766679/real 1775766679]  req@ffff9f1485746300 x1862022114552064/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766704 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766679.563543:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746300 x1862022114552064/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766704 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766679.563548:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766679.563550:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766679.563552:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114552064:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775766679.563578:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114552192:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775766679.563591:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766679.563599:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766679/real 1775766679]  req@ffff9f1485745880 x1862022114552192/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766704 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766679.563607:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745880 x1862022114552192/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766704 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766679.563613:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766679.563616:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766679.563619:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114552192:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766679.563642:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114552320:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775766679.563664:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766679.563677:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766679/real 1775766679]  req@ffff9f1485747480 x1862022114552320/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766704 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766679.563687:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747480 x1862022114552320/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766704 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766679.563692:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766679.563694:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766679.563696:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114552320:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766679.563701:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114552448:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775766679.563716:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766679.563723:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766679/real 1775766679]  req@ffff9f1485744a80 x1862022114552448/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766704 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766679.563730:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744a80 x1862022114552448/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766704 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766679.563740:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766679.563742:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766679.563745:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114552448:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775766679.818695:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766680.586706:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766680.842798:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766681.610695:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775766681.866769:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766682.634687:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775766682.890722:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766683.658726:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766683.914851:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1775766684.682656:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766684.682670:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775766684.682676:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00100000:2.0:1775766684.682686:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775766684.682691:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4062
00000100:00080000:3.0:1775766684.682710:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766684.682720:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775766684.682765:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746680 x1862022114552576/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766684.682797:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766684.682801:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775766684.682803:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766684.682807:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4062
00000100:00080000:3.0:1775766684.682820:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766684.682824:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775766684.682837:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114552576:192.168.202.110@tcp:250:kworker.0
00000008:00100000:3.0:1775766684.682837:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766684.682865:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744700 x1862022114552704/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766684.682886:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766684.682892:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775766684.682895:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766684.682904:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4062
00000100:00080000:3.0:1775766684.682912:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766684.682922:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000800:00000100:2.0:1775766684.682923:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000008:00100000:3.0:1775766684.682938:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766684.682957:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745500 x1862022114552832/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766684.682968:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766684.682972:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775766684.682975:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766684.682980:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4062
00000100:00080000:3.0:1775766684.682985:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00100000:2.0:1775766684.682986:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766684/real 1775766684]  req@ffff9f1485746680 x1862022114552576/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766709 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766684.682998:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766684.683016:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766684.683033:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744000 x1862022114552960/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775766684.683037:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746680 x1862022114552576/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766709 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766684.683093:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766684.683098:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766684.683100:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766684.683111:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114552576:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775766684.683113:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775766684.683131:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:2.0:1775766684.683143:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114552704:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775766684.683143:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4062
00000100:00080000:3.0:1775766684.683156:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766684.683181:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766684.683189:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:2.0:1775766684.683198:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766684.683215:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747800 x1862022114553088/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775766684.683216:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766684/real 1775766684]  req@ffff9f1485744700 x1862022114552704/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766709 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766684.683288:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744700 x1862022114552704/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766709 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766684.683292:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766684.683295:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766684.683304:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00080000:3.0:1775766684.683306:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00100000:2.0:1775766684.683310:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114552704:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775766684.683312:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:2.0:1775766684.683321:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114552832:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775766684.683329:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4062
00000100:00080000:3.0:1775766684.683340:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000800:00000100:2.0:1775766684.683343:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775766684.683355:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775766684.683358:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766684/real 1775766684]  req@ffff9f1485745500 x1862022114552832/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766709 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:3.0:1775766684.683368:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766684.683376:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745500 x1862022114552832/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766709 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766684.683383:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766684.683386:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766684.683395:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114552832:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766684.683397:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745c00 x1862022114553216/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775766684.683404:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114552960:192.168.202.110@tcp:8:kworker.0
00000100:00080000:3.0:1775766684.683430:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000800:00000100:2.0:1775766684.683437:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775766684.683444:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00100000:2.0:1775766684.683449:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766684/real 1775766684]  req@ffff9f1485744000 x1862022114552960/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766709 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766684.683450:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766684.683474:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4062
00000100:00100000:2.0:1775766684.683477:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744000 x1862022114552960/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766709 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766684.683485:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766684.683495:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00080000:2.0:1775766684.683514:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000008:00100000:3.0:1775766684.683515:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00080000:2.0:1775766684.683520:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766684.683527:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114552960:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775766684.683547:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745180 x1862022114553344/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775766684.683550:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114553088:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775766684.683594:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766684.683607:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766684/real 1775766684]  req@ffff9f1485747800 x1862022114553088/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766709 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766684.683614:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766684.683623:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00100000:2.0:1775766684.683628:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747800 x1862022114553088/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766709 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766684.683628:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766684.683649:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766684.683650:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4062
00000100:00080000:2.0:1775766684.683658:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766684.683661:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114553088:192.168.202.110@tcp:8:kworker.0
00000100:00080000:3.0:1775766684.683664:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:2.0:1775766684.683668:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114553216:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775766684.683675:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000800:00000100:2.0:1775766684.683686:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000008:00100000:3.0:1775766684.683686:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775766684.683697:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766684/real 1775766684]  req@ffff9f1485745c00 x1862022114553216/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766709 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766684.683721:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745c00 x1862022114553216/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766709 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766684.683723:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745f80 x1862022114553472/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766684.683762:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766684.683778:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766684.683785:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114553216:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766684.683800:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114553344:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775766684.683813:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766684.683820:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766684/real 1775766684]  req@ffff9f1485745180 x1862022114553344/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766709 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766684.683828:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745180 x1862022114553344/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766709 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766684.683833:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766684.683835:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766684.683837:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114553344:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766684.683841:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114553472:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775766684.683852:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766684.683857:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766684/real 1775766684]  req@ffff9f1485745f80 x1862022114553472/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766709 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766684.683865:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745f80 x1862022114553472/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766709 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766684.683870:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766684.683872:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766684.683874:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114553472:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775766684.938739:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766685.706755:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766685.962850:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766686.730692:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766686.986842:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766687.754688:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766688.010781:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766688.778636:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775766689.034761:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766689.802678:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775766689.802682:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766689.802694:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775766689.802698:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766689.802708:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4067
00000100:00080000:3.0:1775766689.802720:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766689.802726:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775766689.802761:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747100 x1862022114553600/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766689.802786:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766689.802789:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775766689.802791:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766689.802796:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4067
00000100:00080000:3.0:1775766689.802800:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766689.802806:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766689.802816:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766689.802827:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114553600:192.168.202.110@tcp:250:kworker.0
00000100:00100000:3.0:1775766689.802835:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745f80 x1862022114553728/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766689.802852:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766689.802858:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775766689.802860:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766689.802865:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4067
00000100:00080000:3.0:1775766689.802870:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766689.802875:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766689.802880:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766689.802893:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745180 x1862022114553856/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766689.802901:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000800:00000100:2.0:1775766689.802909:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775766689.802912:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775766689.802915:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766689.802921:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4067
00000100:00080000:3.0:1775766689.802926:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766689.802931:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766689.802936:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766689.802950:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745c00 x1862022114553984/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766689.802958:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766689.802962:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775766689.802964:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:2.0:1775766689.802969:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766689/real 1775766689]  req@ffff9f1485747100 x1862022114553600/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766714 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766689.802969:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4067
00000100:00080000:3.0:1775766689.802980:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766689.802995:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766689.803002:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775766689.803014:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747100 x1862022114553600/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766714 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766689.803035:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766689.803038:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766689.803043:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114553600:192.168.202.110@tcp:250:kworker.0
00000100:00100000:3.0:1775766689.803049:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747800 x1862022114554112/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766689.803064:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766689.803069:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775766689.803072:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:2.0:1775766689.803073:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114553728:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775766689.803088:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4067
00000100:00080000:3.0:1775766689.803094:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766689.803101:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000800:00000100:2.0:1775766689.803104:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000008:00100000:3.0:1775766689.803130:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766689.803142:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766689/real 1775766689]  req@ffff9f1485745f80 x1862022114553728/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766714 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766689.803151:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744000 x1862022114554240/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775766689.803155:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745f80 x1862022114553728/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766714 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766689.803195:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766689.803198:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766689.803203:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114553728:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766689.803217:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114553856:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775766689.803218:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766689.803248:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775766689.803252:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766689.803257:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4067
00000800:00000100:2.0:1775766689.803263:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775766689.803263:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766689.803273:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766689.803280:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766689.803287:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766689/real 1775766689]  req@ffff9f1485745180 x1862022114553856/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766714 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766689.803300:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745500 x1862022114554368/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775766689.803305:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745180 x1862022114553856/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766714 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766689.803343:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766689.803344:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766689.803351:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00080000:3.0:1775766689.803356:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00100000:2.0:1775766689.803363:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114553856:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775766689.803363:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766689.803379:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4067
00000100:00100000:2.0:1775766689.803384:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114553984:192.168.202.110@tcp:8:kworker.0
00000100:00080000:3.0:1775766689.803384:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766689.803396:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766689.803405:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766689.803423:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744700 x1862022114554496/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:2.0:1775766689.803424:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766689.803453:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766689/real 1775766689]  req@ffff9f1485745c00 x1862022114553984/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766714 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766689.803469:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745c00 x1862022114553984/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766714 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766689.803479:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766689.803482:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766689.803486:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114553984:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775766689.803492:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114554112:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775766689.803511:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766689.803519:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766689/real 1775766689]  req@ffff9f1485747800 x1862022114554112/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766714 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766689.803527:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747800 x1862022114554112/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766714 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766689.803532:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766689.803534:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766689.803536:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114554112:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775766689.803551:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114554240:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775766689.803587:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766689.803603:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766689/real 1775766689]  req@ffff9f1485744000 x1862022114554240/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766714 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766689.803611:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744000 x1862022114554240/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766714 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766689.803616:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766689.803618:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766689.803620:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114554240:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766689.803624:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114554368:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775766689.803635:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766689.803640:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766689/real 1775766689]  req@ffff9f1485745500 x1862022114554368/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766714 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766689.803647:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745500 x1862022114554368/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766714 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766689.803652:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766689.803654:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766689.803656:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114554368:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766689.803660:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114554496:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775766689.803669:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766689.803675:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766689/real 1775766689]  req@ffff9f1485744700 x1862022114554496/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766714 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766689.803681:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744700 x1862022114554496/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766714 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766689.803686:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766689.803688:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766689.803690:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114554496:192.168.202.110@tcp:8:kworker.0
00010000:00010000:1.0:1775766690.058763:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766690.826749:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766691.082772:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766691.850624:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766692.106750:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766692.874638:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766693.130742:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766693.898657:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766694.154709:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766694.922678:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:2.0:1775766694.922788:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766694.922795:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1775766694.922800:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766694.922811:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4072
00000100:00080000:2.0:1775766694.922825:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766694.922833:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775766694.922876:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0700 x1862022114554624/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766694.922912:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766694.922917:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775766694.922919:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766694.922924:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4072
00000100:00080000:2.0:1775766694.922929:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766694.922935:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766694.922948:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766694.922950:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114554624:192.168.202.110@tcp:250:kworker.0
00000100:00100000:2.0:1775766694.922967:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3800 x1862022114554752/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766694.922984:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766694.922989:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775766694.922992:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766694.922997:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4072
00000100:00080000:2.0:1775766694.923006:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766694.923011:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766694.923018:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000800:00000100:3.0:1775766694.923020:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766694.923041:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2a00 x1862022114554880/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766694.923081:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766694.923085:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775766694.923087:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766694.923092:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4072
00000100:00080000:2.0:1775766694.923097:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766694.923102:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766694.923109:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775766694.923147:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1180 x1862022114555008/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766694.923160:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766694.923163:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1775766694.923165:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766694.923168:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4072
00000100:00080000:2.0:1775766694.923173:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00100000:3.0:1775766694.923175:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766694/real 1775766694]  req@ffff9f14852d0700 x1862022114554624/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766719 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766694.923179:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766694.923187:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775766694.923198:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2680 x1862022114555136/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766694.923207:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766694.923210:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775766694.923214:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:3.0:1775766694.923214:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0700 x1862022114554624/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766719 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766694.923224:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4072
00000100:00080000:2.0:1775766694.923231:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766694.923236:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766694.923239:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766694.923245:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0a80 x1862022114555264/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766694.923269:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766694.923271:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775766694.923272:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766694.923275:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4072
00000100:00080000:2.0:1775766694.923277:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766694.923280:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766694.923284:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766694.923290:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1f80 x1862022114555392/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766694.923296:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766694.923298:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775766694.923299:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766694.923301:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4072
00000100:00080000:2.0:1775766694.923304:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766694.923306:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766694.923309:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775766694.923316:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1500 x1862022114555520/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766694.923358:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766694.923366:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766694.923371:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114554624:192.168.202.110@tcp:250:kworker.0
00000100:00100000:3.0:1775766694.923459:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114554752:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775766694.923488:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766694.923502:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766694/real 1775766694]  req@ffff9f14852d3800 x1862022114554752/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766719 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766694.923514:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3800 x1862022114554752/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766719 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766694.923527:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766694.923530:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766694.923533:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114554752:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766694.923538:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114554880:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775766694.923552:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766694.923606:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766694/real 1775766694]  req@ffff9f14852d2a00 x1862022114554880/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766719 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766694.923638:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2a00 x1862022114554880/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766719 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766694.923646:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766694.923650:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766694.923654:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114554880:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766694.923659:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114555008:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775766694.923677:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766694.923685:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766694/real 1775766694]  req@ffff9f14852d1180 x1862022114555008/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766719 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766694.923694:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1180 x1862022114555008/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766719 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766694.923701:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766694.923703:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766694.923707:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114555008:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775766694.923714:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114555136:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775766694.923736:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766694.923750:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766694/real 1775766694]  req@ffff9f14852d2680 x1862022114555136/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766719 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766694.923781:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2680 x1862022114555136/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766719 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766694.923791:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766694.923795:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766694.923799:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114555136:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775766694.923807:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114555264:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775766694.923830:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766694.923840:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766694/real 1775766694]  req@ffff9f14852d0a80 x1862022114555264/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766719 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766694.923850:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0a80 x1862022114555264/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766719 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766694.923856:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766694.923858:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766694.923861:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114555264:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766694.923865:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114555392:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775766694.923896:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766694.923904:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766694/real 1775766694]  req@ffff9f14852d1f80 x1862022114555392/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766719 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766694.923913:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1f80 x1862022114555392/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766719 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766694.923920:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766694.923922:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766694.923924:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114555392:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766694.923929:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114555520:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775766694.923941:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766694.923948:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766694/real 1775766694]  req@ffff9f14852d1500 x1862022114555520/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766719 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766694.923957:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1500 x1862022114555520/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766719 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766694.923963:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766694.923965:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766694.923968:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114555520:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775766695.178764:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766695.946656:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775766696.202698:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766696.970660:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:0.0:1775766697.226701:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766697.994630:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766698.250739:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766699.018704:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766699.274701:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1775766700.042694:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766700.042703:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1775766700.042707:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00100000:2.0:1775766700.042708:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:0.0:1775766700.042721:0:8:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4077
00000100:00080000:0.0:1775766700.042735:0:8:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766700.042742:0:8:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775766700.042783:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d2300 x1862022114555648/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775766700.042807:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766700.042810:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775766700.042812:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766700.042815:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4077
00000100:00080000:0.0:1775766700.042818:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766700.042821:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775766700.042829:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775766700.042838:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3100 x1862022114555776/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775766700.042845:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766700.042849:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:0.0:1775766700.042850:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766700.042853:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4077
00000100:00080000:0.0:1775766700.042855:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766700.042857:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775766700.042860:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775766700.042866:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3b80 x1862022114555904/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775766700.042873:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114555648:192.168.202.110@tcp:250:kworker.0
00000100:00080000:0.0:1775766700.042876:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766700.042882:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775766700.042885:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766700.042889:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4077
00000100:00080000:0.0:1775766700.042891:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766700.042893:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775766700.042896:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775766700.042904:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d2a00 x1862022114556032/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775766700.042910:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766700.042911:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1775766700.042912:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766700.042915:0:8:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4077
00000100:00080000:0.0:1775766700.042917:0:8:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766700.042920:0:8:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775766700.042923:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775766700.042931:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0a80 x1862022114556160/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775766700.042937:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766700.042939:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775766700.042940:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766700.042943:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4077
00000100:00080000:0.0:1775766700.042946:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766700.042948:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775766700.042952:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775766700.042959:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3800 x1862022114556288/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775766700.042966:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766700.042968:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:0.0:1775766700.042969:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766700.042975:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4077
00000800:00000100:3.0:1775766700.042978:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:0.0:1775766700.042979:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766700.042984:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775766700.042987:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775766700.042994:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d1500 x1862022114556416/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775766700.043000:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766700.043024:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775766700.043025:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766700.043029:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4077
00000100:00080000:0.0:1775766700.043032:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766700.043035:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775766700.043040:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766700.043047:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766700/real 1775766700]  req@ffff9f14a04d2300 x1862022114555648/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766725 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766700.043051:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0000 x1862022114556544/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775766700.043091:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d2300 x1862022114555648/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766725 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766700.043106:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766700.043111:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766700.043116:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114555648:192.168.202.110@tcp:250:kworker.0
00000100:00100000:3.0:1775766700.043124:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114555776:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775766700.043150:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766700.043164:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766700/real 1775766700]  req@ffff9f14a04d3100 x1862022114555776/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766725 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766700.043192:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3100 x1862022114555776/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766725 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766700.043202:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766700.043206:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766700.043210:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114555776:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766700.043236:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114555904:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775766700.043258:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766700.043270:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766700/real 1775766700]  req@ffff9f14a04d3b80 x1862022114555904/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766725 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766700.043284:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3b80 x1862022114555904/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766725 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766700.043292:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766700.043296:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766700.043300:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114555904:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766700.043307:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114556032:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775766700.043328:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766700.043340:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766700/real 1775766700]  req@ffff9f14a04d2a00 x1862022114556032/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766725 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766700.043354:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d2a00 x1862022114556032/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766725 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766700.043362:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766700.043366:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766700.043370:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114556032:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775766700.043376:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114556160:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775766700.043396:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766700.043408:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766700/real 1775766700]  req@ffff9f14a04d0a80 x1862022114556160/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766725 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766700.043420:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0a80 x1862022114556160/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766725 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766700.043428:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766700.043432:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766700.043435:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114556160:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775766700.043442:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114556288:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775766700.043462:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766700.043472:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766700/real 1775766700]  req@ffff9f14a04d3800 x1862022114556288/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766725 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766700.043485:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3800 x1862022114556288/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766725 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766700.043494:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766700.043497:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766700.043500:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114556288:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766700.043506:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114556416:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775766700.043525:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766700.043536:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766700/real 1775766700]  req@ffff9f14a04d1500 x1862022114556416/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766725 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766700.043549:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1500 x1862022114556416/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766725 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766700.043586:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766700.043590:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766700.043593:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114556416:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766700.043599:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114556544:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775766700.043620:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766700.043631:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766700/real 1775766700]  req@ffff9f14a04d0000 x1862022114556544/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766725 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766700.043644:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0000 x1862022114556544/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766725 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766700.043659:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766700.043663:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766700.043666:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114556544:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775766700.298777:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766701.066678:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766701.322683:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766702.090707:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766702.346776:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766703.114676:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775766703.370684:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766704.138673:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:0.0:1775766704.394753:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1775766705.162658:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766705.162666:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1775766705.162671:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00100000:2.0:1775766705.162721:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:0.0:1775766705.162754:0:8:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4082
00000100:00080000:0.0:1775766705.162766:0:8:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766705.162773:0:8:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775766705.162809:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3480 x1862022114556672/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775766705.162838:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766705.162841:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775766705.162842:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766705.162845:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4082
00000100:00080000:0.0:1775766705.162848:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766705.162851:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775766705.162859:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775766705.162869:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d1180 x1862022114556800/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775766705.162877:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766705.162880:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:0.0:1775766705.162881:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766705.162884:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4082
00000100:00080000:0.0:1775766705.162887:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00100000:3.0:1775766705.162889:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114556672:192.168.202.110@tcp:250:kworker.0
00000100:00080000:0.0:1775766705.162892:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775766705.162900:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775766705.162909:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0e00 x1862022114556928/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775766705.162917:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766705.162919:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775766705.162921:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766705.162924:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4082
00000100:00080000:0.0:1775766705.162926:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766705.162928:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775766705.162932:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775766705.162940:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d1f80 x1862022114557056/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775766705.162947:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766705.162966:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1775766705.162967:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766705.162970:0:8:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4082
00000100:00080000:0.0:1775766705.162986:0:8:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000800:00000100:3.0:1775766705.162990:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:0.0:1775766705.162992:0:8:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775766705.162998:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775766705.163009:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0380 x1862022114557184/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775766705.163017:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766705.163019:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775766705.163022:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766705.163025:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4082
00000100:00080000:0.0:1775766705.163029:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766705.163032:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775766705.163036:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766705/real 1775766705]  req@ffff9f14a04d3480 x1862022114556672/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766730 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:0.0:1775766705.163037:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775766705.163053:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0000 x1862022114557312/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775766705.163060:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3480 x1862022114556672/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766730 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766705.163063:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766705.163071:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:0.0:1775766705.163074:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766705.163080:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4082
00000100:00080000:3.0:1775766705.163081:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766705.163082:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766705.163087:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:0.0:1775766705.163088:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775766705.163094:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766705.163094:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114556672:192.168.202.110@tcp:250:kworker.0
00000100:00100000:0.0:1775766705.163105:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d1500 x1862022114557440/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775766705.163105:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114556800:192.168.202.110@tcp:38:kworker.0
00000100:00080000:0.0:1775766705.163119:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766705.163122:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775766705.163124:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766705.163128:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4082
00000100:00080000:0.0:1775766705.163131:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000800:00000100:3.0:1775766705.163135:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:0.0:1775766705.163137:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775766705.163142:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766705.163149:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766705/real 1775766705]  req@ffff9f14a04d1180 x1862022114556800/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766730 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766705.163154:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3800 x1862022114557568/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775766705.163170:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1180 x1862022114556800/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766730 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766705.163177:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766705.163180:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766705.163185:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114556800:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766705.163226:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114556928:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775766705.163241:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766705.163248:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766705/real 1775766705]  req@ffff9f14a04d0e00 x1862022114556928/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766730 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766705.163256:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0e00 x1862022114556928/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766730 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766705.163262:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766705.163264:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766705.163266:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114556928:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766705.163270:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114557056:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775766705.163280:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766705.163286:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766705/real 1775766705]  req@ffff9f14a04d1f80 x1862022114557056/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766730 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766705.163293:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1f80 x1862022114557056/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766730 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766705.163298:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766705.163300:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766705.163302:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114557056:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775766705.163305:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114557184:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775766705.163315:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766705.163321:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766705/real 1775766705]  req@ffff9f14a04d0380 x1862022114557184/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766730 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766705.163328:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0380 x1862022114557184/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766730 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766705.163333:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766705.163335:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766705.163337:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114557184:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775766705.163340:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114557312:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775766705.163350:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766705.163356:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766705/real 1775766705]  req@ffff9f14a04d0000 x1862022114557312/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766730 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766705.163364:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0000 x1862022114557312/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766730 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766705.163369:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766705.163371:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766705.163373:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114557312:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766705.163376:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114557440:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775766705.163386:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766705.163391:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766705/real 1775766705]  req@ffff9f14a04d1500 x1862022114557440/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766730 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766705.163399:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1500 x1862022114557440/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766730 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766705.163404:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766705.163406:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766705.163408:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114557440:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766705.163412:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114557568:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775766705.163421:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766705.163426:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766705/real 1775766705]  req@ffff9f14a04d3800 x1862022114557568/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766730 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766705.163433:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3800 x1862022114557568/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766730 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766705.163438:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766705.163440:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766705.163442:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114557568:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775766705.418706:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766706.186655:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766706.442768:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766707.210626:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766707.466780:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766708.234672:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766708.490807:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766709.258678:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766709.514715:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1775766710.282693:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:2.0:1775766710.282700:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:0.0:1775766710.282706:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1775766710.282714:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766710.282729:0:8:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4087
00000100:00080000:0.0:1775766710.282745:0:8:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766710.282757:0:8:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775766710.282817:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0a80 x1862022114557696/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775766710.282861:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766710.282867:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775766710.282869:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766710.282874:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4087
00000100:00080000:0.0:1775766710.282879:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766710.282883:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775766710.282894:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775766710.282926:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d2a00 x1862022114557824/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775766710.282961:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766710.282964:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:0.0:1775766710.282966:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766710.282969:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4087
00000100:00080000:0.0:1775766710.282972:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766710.282976:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775766710.282980:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775766710.282992:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3b80 x1862022114557952/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775766710.283006:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114557696:192.168.202.110@tcp:250:kworker.0
00000100:00080000:0.0:1775766710.283007:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766710.283024:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775766710.283027:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766710.283031:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4087
00000100:00080000:0.0:1775766710.283033:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766710.283036:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775766710.283040:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775766710.283054:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3100 x1862022114558080/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775766710.283061:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766710.283069:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1775766710.283071:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766710.283077:0:8:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4087
00000100:00080000:0.0:1775766710.283079:0:8:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766710.283082:0:8:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775766710.283086:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775766710.283095:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d2300 x1862022114558208/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:3.0:1775766710.283107:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:0.0:1775766710.283118:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766710.283123:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775766710.283126:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766710.283135:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4087
00000100:00080000:0.0:1775766710.283141:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766710.283148:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775766710.283158:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766710.283162:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766710/real 1775766710]  req@ffff9f14a04d0a80 x1862022114557696/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766735 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766710.283182:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3800 x1862022114558336/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775766710.283188:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0a80 x1862022114557696/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766735 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766710.283223:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766710.283225:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:0.0:1775766710.283244:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766710.283247:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766710.283255:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4087
00000100:00080000:3.0:1775766710.283261:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:0.0:1775766710.283264:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:3.0:1775766710.283271:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114557696:192.168.202.110@tcp:250:kworker.0
00000100:00080000:0.0:1775766710.283279:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775766710.283289:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766710.283292:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114557824:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766710.283326:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d1500 x1862022114558464/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:3.0:1775766710.283340:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:0.0:1775766710.283342:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766710.283348:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775766710.283350:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:3.0:1775766710.283354:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766710/real 1775766710]  req@ffff9f14a04d2a00 x1862022114557824/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766735 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766710.283355:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4087
00000100:00080000:0.0:1775766710.283366:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766710.283377:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775766710.283383:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d2a00 x1862022114557824/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766735 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:0.0:1775766710.283388:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00080000:3.0:1775766710.283404:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00100000:0.0:1775766710.283414:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0000 x1862022114558592/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766710.283414:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766710.283429:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114557824:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766710.283474:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114557952:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775766710.283494:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766710.283534:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766710/real 1775766710]  req@ffff9f14a04d3b80 x1862022114557952/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766735 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766710.283543:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3b80 x1862022114557952/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766735 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766710.283548:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766710.283551:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766710.283590:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114557952:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766710.283595:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114558080:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775766710.283610:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766710.283617:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766710/real 1775766710]  req@ffff9f14a04d3100 x1862022114558080/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766735 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766710.283625:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3100 x1862022114558080/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766735 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766710.283631:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766710.283633:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766710.283635:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114558080:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775766710.283638:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114558208:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775766710.283648:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766710.283653:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766710/real 1775766710]  req@ffff9f14a04d2300 x1862022114558208/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766735 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766710.283660:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d2300 x1862022114558208/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766735 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766710.283665:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766710.283667:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766710.283669:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114558208:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775766710.283672:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114558336:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775766710.283682:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766710.283688:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766710/real 1775766710]  req@ffff9f14a04d3800 x1862022114558336/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766735 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766710.283695:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3800 x1862022114558336/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766735 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766710.283705:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766710.283707:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766710.283709:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114558336:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766710.283712:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114558464:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775766710.283722:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766710.283728:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766710/real 1775766710]  req@ffff9f14a04d1500 x1862022114558464/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766735 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766710.283735:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1500 x1862022114558464/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766735 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766710.283740:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766710.283742:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766710.283744:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114558464:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766710.283748:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114558592:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775766710.283757:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766710.283762:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766710/real 1775766710]  req@ffff9f14a04d0000 x1862022114558592/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766735 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766710.283768:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0000 x1862022114558592/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766735 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766710.283773:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766710.283776:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766710.283778:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114558592:192.168.202.110@tcp:8:kworker.0
00010000:00010000:3.0:1775766710.538948:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766711.306624:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:0.0:1775766711.562748:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766712.330655:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766712.586691:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766713.354647:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766713.610715:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766714.378636:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766714.634687:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766715.402693:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:0.0:1775766715.402711:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766715.402720:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1775766715.402725:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766715.402754:0:8:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4092
00000100:00080000:0.0:1775766715.402769:0:8:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766715.402789:0:8:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775766715.402901:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0380 x1862022114558720/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775766715.402932:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766715.402935:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775766715.402936:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766715.402940:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4092
00000100:00080000:0.0:1775766715.402944:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766715.402947:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775766715.402957:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775766715.402972:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d1f80 x1862022114558848/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775766715.402980:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766715.402983:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:0.0:1775766715.402985:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766715.402990:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4092
00000100:00080000:0.0:1775766715.402993:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766715.402998:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775766715.403007:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766715.403007:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114558720:192.168.202.110@tcp:250:kworker.0
00000100:00100000:0.0:1775766715.403021:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0e00 x1862022114558976/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775766715.403030:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766715.403032:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775766715.403033:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766715.403036:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4092
00000100:00080000:0.0:1775766715.403038:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766715.403041:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775766715.403044:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775766715.403053:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d1180 x1862022114559104/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775766715.403060:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766715.403063:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1775766715.403065:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766715.403070:0:8:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4092
00000100:00080000:0.0:1775766715.403075:0:8:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766715.403079:0:8:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775766715.403085:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775766715.403098:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3480 x1862022114559232/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775766715.403110:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766715.403115:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000800:00000100:3.0:1775766715.403121:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:0.0:1775766715.403123:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766715.403131:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4092
00000100:00080000:0.0:1775766715.403138:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766715.403155:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775766715.403160:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775766715.403173:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0000 x1862022114559360/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775766715.403180:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766715.403183:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:0.0:1775766715.403186:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766715.403190:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4092
00000100:00080000:0.0:1775766715.403195:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766715.403198:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775766715.403205:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766715.403217:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766715/real 1775766715]  req@ffff9f14a04d0380 x1862022114558720/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766740 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766715.403218:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d1500 x1862022114559488/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775766715.403288:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766715.403292:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775766715.403293:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766715.403300:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4092
00000100:00080000:0.0:1775766715.403304:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:3.0:1775766715.403310:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0380 x1862022114558720/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766740 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766715.403315:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775766715.403319:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775766715.403341:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3800 x1862022114559616/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766715.403379:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766715.403387:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766715.403395:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114558720:192.168.202.110@tcp:250:kworker.0
00000100:00100000:3.0:1775766715.403411:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114558848:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775766715.403440:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766715.403457:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766715/real 1775766715]  req@ffff9f14a04d1f80 x1862022114558848/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766740 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766715.403475:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1f80 x1862022114558848/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766740 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766715.403489:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766715.403493:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766715.403499:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114558848:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766715.403538:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114558976:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775766715.403613:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766715.403625:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766715/real 1775766715]  req@ffff9f14a04d0e00 x1862022114558976/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766740 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766715.403655:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0e00 x1862022114558976/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766740 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766715.403663:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766715.403678:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766715.403681:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114558976:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766715.403688:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114559104:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775766715.403708:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766715.403717:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766715/real 1775766715]  req@ffff9f14a04d1180 x1862022114559104/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766740 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766715.403728:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1180 x1862022114559104/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766740 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766715.403736:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766715.403740:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766715.403744:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114559104:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775766715.403759:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114559232:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775766715.403801:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766715.403823:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766715/real 1775766715]  req@ffff9f14a04d3480 x1862022114559232/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766740 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766715.403877:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3480 x1862022114559232/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766740 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766715.403885:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766715.403888:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766715.403891:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114559232:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775766715.403907:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114559360:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775766715.403928:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766715.403938:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766715/real 1775766715]  req@ffff9f14a04d0000 x1862022114559360/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766740 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766715.403948:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0000 x1862022114559360/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766740 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766715.403954:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766715.403957:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766715.403959:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114559360:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766715.403964:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114559488:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775766715.403979:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766715.403986:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766715/real 1775766715]  req@ffff9f14a04d1500 x1862022114559488/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766740 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766715.403996:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1500 x1862022114559488/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766740 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766715.404008:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766715.404011:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766715.404016:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114559488:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766715.404024:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114559616:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775766715.404048:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766715.404061:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766715/real 1775766715]  req@ffff9f14a04d3800 x1862022114559616/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766740 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766715.404079:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3800 x1862022114559616/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766740 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766715.404091:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766715.404095:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766715.404099:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114559616:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775766715.658772:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766716.426695:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766716.682848:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766717.450655:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775766717.706808:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766718.474629:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:0.0:1775766718.730694:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766719.498732:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766719.754751:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766720.522644:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775766720.522747:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766720.522759:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775766720.522766:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766720.522815:0:8:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4098
00000100:00080000:3.0:1775766720.522849:0:8:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766720.522858:0:8:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775766720.522911:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746680 x1862022114559744/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766720.522952:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766720.522957:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775766720.522959:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766720.522964:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4098
00000100:00080000:3.0:1775766720.522969:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766720.522974:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766720.522986:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766720.522998:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114559744:192.168.202.110@tcp:250:kworker.0
00000100:00100000:3.0:1775766720.523002:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744a80 x1862022114559872/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766720.523024:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766720.523028:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775766720.523031:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766720.523056:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4098
00000100:00080000:3.0:1775766720.523061:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766720.523066:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766720.523074:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766720.523089:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747480 x1862022114560000/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766720.523100:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000800:00000100:2.0:1775766720.523112:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775766720.523116:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775766720.523120:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766720.523129:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4098
00000100:00080000:3.0:1775766720.523138:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766720.523145:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766720.523154:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775766720.523168:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766720/real 1775766720]  req@ffff9f1485746680 x1862022114559744/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766745 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766720.523173:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745880 x1862022114560128/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766720.523213:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766720.523224:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775766720.523226:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:2.0:1775766720.523236:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746680 x1862022114559744/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766745 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766720.523243:0:8:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4098
00000100:00080000:3.0:1775766720.523256:0:8:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766720.523270:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766720.523271:0:8:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766720.523284:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00080000:2.0:1775766720.523285:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766720.523296:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114559744:192.168.202.110@tcp:250:kworker.0
00000100:00100000:3.0:1775766720.523310:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746300 x1862022114560256/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766720.523327:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766720.523334:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00100000:2.0:1775766720.523335:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114559872:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775766720.523341:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766720.523364:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4098
00000100:00080000:3.0:1775766720.523371:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000800:00000100:2.0:1775766720.523381:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775766720.523389:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766720.523399:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766720.523405:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766720/real 1775766720]  req@ffff9f1485744a80 x1862022114559872/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766745 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766720.523418:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746a00 x1862022114560384/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775766720.523424:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744a80 x1862022114559872/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766745 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766720.523462:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766720.523477:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775766720.523484:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766720.523495:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4098
00000100:00080000:2.0:1775766720.523498:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766720.523500:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766720.523512:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00080000:2.0:1775766720.523516:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000008:00100000:3.0:1775766720.523527:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766720.523534:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114559872:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766720.523548:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744380 x1862022114560512/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775766720.523550:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114560000:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775766720.523602:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766720.523630:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766720/real 1775766720]  req@ffff9f1485747480 x1862022114560000/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766745 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766720.523640:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747480 x1862022114560000/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766745 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766720.523698:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766720.523716:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766720.523724:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114560000:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775766720.523724:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766720.523745:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775766720.523749:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:2.0:1775766720.523754:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114560128:192.168.202.110@tcp:8:kworker.0
00000100:00080000:3.0:1775766720.523764:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4098
00000100:00080000:3.0:1775766720.523775:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766720.523781:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000800:00000100:2.0:1775766720.523805:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000008:00100000:3.0:1775766720.523829:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775766720.523835:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766720/real 1775766720]  req@ffff9f1485745880 x1862022114560128/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766745 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766720.523849:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744e00 x1862022114560640/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775766720.523856:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745880 x1862022114560128/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766745 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766720.523876:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766720.523882:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766720.523886:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114560128:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775766720.523892:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114560256:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775766720.523912:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766720.523920:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766720/real 1775766720]  req@ffff9f1485746300 x1862022114560256/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766745 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766720.523929:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746300 x1862022114560256/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766745 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766720.523935:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766720.523937:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766720.523939:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114560256:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775766720.523953:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114560384:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775766720.523964:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766720.523970:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766720/real 1775766720]  req@ffff9f1485746a00 x1862022114560384/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766745 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766720.523977:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746a00 x1862022114560384/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766745 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766720.523982:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766720.523984:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766720.523986:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114560384:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766720.523990:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114560512:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775766720.524000:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766720.524006:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766720/real 1775766720]  req@ffff9f1485744380 x1862022114560512/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766745 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766720.524013:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744380 x1862022114560512/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766745 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766720.524018:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766720.524021:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766720.524023:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114560512:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766720.524027:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114560640:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775766720.524036:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766720.524041:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766720/real 1775766720]  req@ffff9f1485744e00 x1862022114560640/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766745 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766720.524047:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744e00 x1862022114560640/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766745 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766720.524062:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766720.524064:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766720.524066:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114560640:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775766720.778702:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766721.546653:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766721.802737:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766722.570652:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766722.826903:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766723.594793:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766723.850871:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766724.618663:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775766724.874794:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1775766725.642641:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766725.642650:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775766725.642654:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00100000:2.0:1775766725.642665:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775766725.642666:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4103
00000100:00080000:3.0:1775766725.642698:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766725.642704:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775766725.642772:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747b80 x1862022114560768/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766725.642799:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766725.642817:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775766725.642819:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766725.642822:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4103
00000100:00080000:3.0:1775766725.642825:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766725.642829:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766725.642839:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766725.642858:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744e00 x1862022114560896/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775766725.642859:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114560768:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775766725.642881:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766725.642884:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775766725.642885:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766725.642890:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4103
00000100:00080000:3.0:1775766725.642892:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766725.642895:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766725.642898:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766725.642906:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744380 x1862022114561024/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766725.642913:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766725.642915:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775766725.642917:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766725.642921:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4103
00000100:00080000:3.0:1775766725.642931:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766725.642935:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766725.642939:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766725.642948:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746a00 x1862022114561152/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:2.0:1775766725.642956:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775766725.642965:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766725.642967:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775766725.642969:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766725.642973:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4103
00000100:00080000:3.0:1775766725.642976:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766725.642979:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766725.642984:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766725.642994:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746300 x1862022114561280/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766725.643002:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766725.643020:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775766725.643022:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766725.643028:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4103
00000100:00080000:3.0:1775766725.643033:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:2.0:1775766725.643035:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766725/real 1775766725]  req@ffff9f1485747b80 x1862022114560768/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766750 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766725.643041:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766725.643057:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766725.643076:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747b80 x1862022114560768/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766750 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766725.643077:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745880 x1862022114561408/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766725.643116:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766725.643129:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766725.643138:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114560768:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775766725.643138:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766725.643148:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775766725.643156:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766725.643162:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4103
00000100:00080000:3.0:1775766725.643166:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766725.643170:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766725.643174:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766725.643186:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114560896:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766725.643192:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747480 x1862022114561536/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766725.643202:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766725.643206:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775766725.643207:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766725.643211:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4103
00000100:00080000:3.0:1775766725.643216:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000800:00000100:2.0:1775766725.643219:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775766725.643229:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775766725.643233:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766725/real 1775766725]  req@ffff9f1485744e00 x1862022114560896/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766750 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:3.0:1775766725.643258:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775766725.643259:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744e00 x1862022114560896/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766750 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766725.643273:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766725.643279:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766725.643280:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744a80 x1862022114561664/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775766725.643292:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114560896:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766725.643319:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114561024:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775766725.643343:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766725.643354:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766725/real 1775766725]  req@ffff9f1485744380 x1862022114561024/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766750 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766725.643362:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744380 x1862022114561024/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766750 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766725.643373:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766725.643375:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766725.643377:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114561024:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766725.643393:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114561152:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775766725.643405:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766725.643410:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766725/real 1775766725]  req@ffff9f1485746a00 x1862022114561152/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766750 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766725.643419:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746a00 x1862022114561152/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766750 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766725.643428:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766725.643431:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766725.643434:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114561152:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775766725.643440:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114561280:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775766725.643463:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766725.643474:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766725/real 1775766725]  req@ffff9f1485746300 x1862022114561280/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766750 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766725.643484:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746300 x1862022114561280/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766750 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766725.643490:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766725.643492:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766725.643495:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114561280:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775766725.643499:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114561408:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775766725.643514:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766725.643521:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766725/real 1775766725]  req@ffff9f1485745880 x1862022114561408/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766750 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766725.643528:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745880 x1862022114561408/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766750 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766725.643534:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766725.643536:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766725.643538:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114561408:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766725.643552:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114561536:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775766725.643586:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766725.643593:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766725/real 1775766725]  req@ffff9f1485747480 x1862022114561536/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766750 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766725.643601:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747480 x1862022114561536/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766750 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766725.643606:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766725.643608:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766725.643610:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114561536:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766725.643614:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114561664:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775766725.643625:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766725.643631:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766725/real 1775766725]  req@ffff9f1485744a80 x1862022114561664/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766750 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766725.643638:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744a80 x1862022114561664/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766750 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766725.643643:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766725.643645:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766725.643647:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114561664:192.168.202.110@tcp:8:kworker.0
00010000:00010000:0.0:1775766725.898763:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766726.666674:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766726.922883:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766727.690740:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766727.946794:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766728.714788:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766728.970782:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766729.738757:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766729.994742:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1775766730.762652:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766730.762663:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1775766730.762668:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775766730.762671:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:1.0:1775766730.762693:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4108
00000100:00080000:1.0:1775766730.762718:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766730.762727:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775766730.762765:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff100 x1862022114561792/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766730.762805:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766730.762809:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775766730.762812:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766730.762818:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4108
00000100:00080000:1.0:1775766730.762824:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766730.762829:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766730.762840:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775766730.762847:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114561792:192.168.202.110@tcp:250:kworker.0
00000100:00100000:1.0:1775766730.762866:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fd880 x1862022114561920/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766730.762880:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766730.762885:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775766730.762888:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766730.762893:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4108
00000100:00080000:1.0:1775766730.762907:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766730.762912:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766730.762919:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775766730.762935:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fe300 x1862022114562048/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766730.762947:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766730.762951:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775766730.762953:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775766730.762959:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114561920:192.168.202.110@tcp:38:kworker.0
00000100:00080000:1.0:1775766730.762962:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4108
00000100:00080000:1.0:1775766730.762970:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766730.762975:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766730.762982:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775766730.762995:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114562048:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775766730.762999:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fc380 x1862022114562176/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766730.763025:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766730.763029:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1775766730.763031:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775766730.763036:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114562176:192.168.202.110@tcp:8:kworker.0
00000100:00080000:1.0:1775766730.763039:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4108
00000100:00080000:1.0:1775766730.763046:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766730.763051:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766730.763058:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775766730.763073:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fce00 x1862022114562304/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766730.763092:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766730.763097:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775766730.763099:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775766730.763106:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114562304:192.168.202.110@tcp:8:kworker.0
00000100:00080000:1.0:1775766730.763107:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4108
00000100:00080000:1.0:1775766730.763115:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766730.763121:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766730.763128:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775766730.763143:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fca80 x1862022114562432/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766730.763162:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766730.763166:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775766730.763168:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766730.763174:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4108
00000100:00080000:1.0:1775766730.763180:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1775766730.763184:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114562432:192.168.202.110@tcp:38:kworker.0
00000100:00080000:1.0:1775766730.763187:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766730.763196:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775766730.763211:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fed80 x1862022114562560/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766730.763224:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766730.763228:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775766730.763230:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775766730.763233:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114562560:192.168.202.110@tcp:38:kworker.0
00000100:00080000:1.0:1775766730.763240:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4108
00000100:00080000:1.0:1775766730.763246:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766730.763250:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766730.763257:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775766730.763271:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdc00 x1862022114562688/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775766730.763310:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114562688:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775766731.018742:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766731.786634:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775766732.042744:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766732.810626:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:0.0:1775766733.066650:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766733.834669:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000400:00000100:2.0:1775766733.834898:0:2390:0:(lib-socket.c:454:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.202.110:988
00000400:00000100:2.0:1775766733.834950:0:2390:0:(acceptor.c:111:lnet_connect_console_error()) Connection to 192.168.202.110@tcp at host 192.168.202.110 was unreachable: the network or that node may be down, or Lustre may be misconfigured.
00000800:00000100:2.0:1775766733.834972:0:2390:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:2.0:1775766733.835027:0:2390:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:2.0:1775766733.835064:0:2390:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:2.0:1775766733.835076:0:2390:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:2.0:1775766733.835088:0:2390:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:2.0:1775766733.835099:0:2390:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000100:00100000:0.0:1775766733.835103:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766730/real 1775766733]  req@ffff9f14ab4fc380 x1862022114562176/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766755 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000800:00000100:2.0:1775766733.835113:0:2390:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:2.0:1775766733.835130:0:2390:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000100:00100000:0.0:1775766733.835154:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fc380 x1862022114562176/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766755 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766733.835169:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766733.835175:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766733.835180:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114562176:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775766733.835198:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766730/real 1775766733]  req@ffff9f14ab4fd880 x1862022114561920/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766755 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766733.835214:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fd880 x1862022114561920/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766755 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766733.835237:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766733.835241:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766733.835246:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114561920:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766733.835254:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766730/real 1775766733]  req@ffff9f14ab4fe300 x1862022114562048/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766755 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766733.835269:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fe300 x1862022114562048/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766755 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766733.835278:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766733.835283:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766733.835286:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114562048:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766733.835299:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766730/real 1775766733]  req@ffff9f14ab4ff100 x1862022114561792/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766755 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766733.835318:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff100 x1862022114561792/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766755 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766733.835326:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766733.835329:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766733.835333:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114561792:192.168.202.110@tcp:250:kworker.0
00000100:00100000:0.0:1775766733.835371:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766730/real 1775766733]  req@ffff9f14ab4fdc00 x1862022114562688/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766755 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766733.835385:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdc00 x1862022114562688/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766755 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766733.835394:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766733.835399:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766733.835403:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114562688:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775766733.835410:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766730/real 1775766733]  req@ffff9f14ab4fed80 x1862022114562560/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766755 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766733.835423:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fed80 x1862022114562560/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766755 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766733.835431:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766733.835436:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766733.835438:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114562560:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766733.835446:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766730/real 1775766733]  req@ffff9f14ab4fca80 x1862022114562432/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766755 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766733.835461:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fca80 x1862022114562432/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766755 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766733.835468:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766733.835472:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766733.835476:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114562432:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766733.835482:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766730/real 1775766733]  req@ffff9f14ab4fce00 x1862022114562304/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766755 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766733.835494:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fce00 x1862022114562304/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766755 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766733.835501:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766733.835504:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766733.835506:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114562304:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775766734.090732:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766734.858633:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766735.114761:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766735.882643:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775766735.882673:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766735.882681:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775766735.882686:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766735.882696:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4113
00000100:00080000:3.0:1775766735.882709:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766735.882715:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775766735.882748:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746680 x1862022114562816/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766735.882771:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766735.882774:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775766735.882775:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766735.882779:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4113
00000100:00080000:3.0:1775766735.882781:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766735.882785:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766735.882801:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775766735.882810:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114562816:192.168.202.110@tcp:250:kworker.0
00000100:00100000:3.0:1775766735.882817:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744700 x1862022114562944/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766735.882830:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766735.882833:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775766735.882835:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766735.882840:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4113
00000100:00080000:3.0:1775766735.882844:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766735.882848:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766735.882854:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766735.882869:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745500 x1862022114563072/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766735.882881:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766735.882887:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775766735.882889:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766735.882894:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4113
00000100:00080000:3.0:1775766735.882899:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000800:00000100:0.0:1775766735.882903:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775766735.882907:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766735.882913:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766735.882925:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744000 x1862022114563200/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766735.882935:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766735.882939:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775766735.882941:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766735.882946:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4113
00000100:00080000:3.0:1775766735.882951:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1775766735.882956:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766735/real 1775766735]  req@ffff9f1485746680 x1862022114562816/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766760 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766735.882959:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766735.882969:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766735.882985:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747800 x1862022114563328/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775766735.882991:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746680 x1862022114562816/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766760 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766735.883001:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766735.883009:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775766735.883013:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766735.883018:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766735.883021:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4113
00000100:00080000:0.0:1775766735.883028:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:3.0:1775766735.883029:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1775766735.883036:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114562816:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775766735.883038:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766735.883049:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766735.883062:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745c00 x1862022114563456/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775766735.883071:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114562944:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775766735.883086:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766735.883089:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775766735.883091:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766735.883096:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4113
00000100:00080000:3.0:1775766735.883102:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766735.883107:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000800:00000100:0.0:1775766735.883112:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000008:00100000:3.0:1775766735.883114:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775766735.883129:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766735/real 1775766735]  req@ffff9f1485744700 x1862022114562944/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766760 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766735.883141:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745180 x1862022114563584/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775766735.883146:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744700 x1862022114562944/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766760 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766735.883167:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766735.883169:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766735.883173:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766735.883179:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114562944:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775766735.883180:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775766735.883184:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775766735.883190:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114563072:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775766735.883192:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4113
00000100:00080000:3.0:1775766735.883201:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766735.883206:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766735.883213:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766735.883227:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745f80 x1862022114563712/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1775766735.883231:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766735.883248:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766735/real 1775766735]  req@ffff9f1485745500 x1862022114563072/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766760 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766735.883262:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745500 x1862022114563072/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766760 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766735.883271:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766735.883275:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766735.883280:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114563072:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766735.883287:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114563200:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766735.883308:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766735.883317:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766735/real 1775766735]  req@ffff9f1485744000 x1862022114563200/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766760 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766735.883328:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744000 x1862022114563200/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766760 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766735.883336:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766735.883347:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766735.883350:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114563200:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775766735.883357:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114563328:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766735.883379:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766735.883389:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766735/real 1775766735]  req@ffff9f1485747800 x1862022114563328/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766760 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766735.883400:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747800 x1862022114563328/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766760 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766735.883407:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766735.883410:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766735.883413:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114563328:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775766735.883431:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114563456:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766735.883453:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766735.883464:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766735/real 1775766735]  req@ffff9f1485745c00 x1862022114563456/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766760 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766735.883481:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745c00 x1862022114563456/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766760 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766735.883489:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766735.883492:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766735.883495:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114563456:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766735.883501:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114563584:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766735.883521:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766735.883532:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766735/real 1775766735]  req@ffff9f1485745180 x1862022114563584/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766760 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766735.883545:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745180 x1862022114563584/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766760 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766735.883580:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766735.883584:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766735.883588:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114563584:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766735.883595:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114563712:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766735.883615:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766735.883625:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766735/real 1775766735]  req@ffff9f1485745f80 x1862022114563712/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766760 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766735.883657:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745f80 x1862022114563712/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766760 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766735.883672:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766735.883676:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766735.883679:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114563712:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775766736.138745:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766736.906670:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766737.162742:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766737.930671:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766738.186771:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766738.954685:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775766739.210774:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766739.978664:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:0.0:1775766740.234761:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1775766741.002691:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1775766741.002697:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:2.0:1775766741.002700:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1775766741.002706:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766741.002715:0:8:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4118
00000100:00080000:2.0:1775766741.002726:0:8:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766741.002732:0:8:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775766741.002766:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0380 x1862022114563840/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766741.002788:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766741.002790:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775766741.002792:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766741.002795:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4118
00000100:00080000:2.0:1775766741.002798:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766741.002800:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766741.002808:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766741.002817:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2d80 x1862022114563968/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766741.002835:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766741.002839:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00100000:0.0:1775766741.002842:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114563840:192.168.202.110@tcp:250:kworker.0
00000100:00080000:2.0:1775766741.002842:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766741.002849:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4118
00000100:00080000:2.0:1775766741.002853:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766741.002856:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766741.002859:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766741.002868:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0000 x1862022114564096/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766741.002875:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766741.002877:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775766741.002879:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766741.002882:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4118
00000100:00080000:2.0:1775766741.002884:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766741.002886:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766741.002896:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775766741.002903:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3b80 x1862022114564224/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766741.002909:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766741.002911:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1775766741.002912:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766741.002915:0:8:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4118
00000100:00080000:2.0:1775766741.002918:0:8:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766741.002922:0:8:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000800:00000100:0.0:1775766741.002924:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000008:00100000:2.0:1775766741.002926:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775766741.002934:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0e00 x1862022114564352/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766741.002942:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766741.002945:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775766741.002946:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766741.002949:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4118
00000100:00080000:2.0:1775766741.002951:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766741.002954:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766741.002957:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766741.002964:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1880 x1862022114564480/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766741.002975:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766741.002977:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775766741.002979:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766741.002981:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4118
00000100:00080000:2.0:1775766741.002984:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766741.002989:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775766741.002990:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766741/real 1775766741]  req@ffff9f14852d0380 x1862022114563840/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766766 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:2.0:1775766741.002991:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766741.003020:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3100 x1862022114564608/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766741.003029:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766741.003032:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775766741.003034:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775766741.003040:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0380 x1862022114563840/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766766 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766741.003041:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4118
00000100:00080000:2.0:1775766741.003050:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766741.003057:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766741.003082:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00080000:0.0:1775766741.003084:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766741.003092:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766741.003093:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2300 x1862022114564736/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775766741.003102:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114563840:192.168.202.110@tcp:250:kworker.0
00000100:00100000:0.0:1775766741.003129:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114563968:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766741.003154:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766741.003169:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766741/real 1775766741]  req@ffff9f14852d2d80 x1862022114563968/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766766 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766741.003191:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2d80 x1862022114563968/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766766 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766741.003200:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766741.003204:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766741.003207:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114563968:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766741.003232:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114564096:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766741.003270:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766741.003300:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766741/real 1775766741]  req@ffff9f14852d0000 x1862022114564096/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766766 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766741.003313:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0000 x1862022114564096/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766766 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766741.003321:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766741.003325:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766741.003328:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114564096:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766741.003334:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114564224:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766741.003355:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766741.003365:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766741/real 1775766741]  req@ffff9f14852d3b80 x1862022114564224/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766766 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766741.003376:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3b80 x1862022114564224/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766766 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766741.003389:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766741.003392:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766741.003395:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114564224:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775766741.003401:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114564352:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766741.003416:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766741.003425:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766741/real 1775766741]  req@ffff9f14852d0e00 x1862022114564352/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766766 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766741.003435:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0e00 x1862022114564352/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766766 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766741.003443:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766741.003446:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766741.003449:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114564352:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775766741.003455:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114564480:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766741.003472:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766741.003482:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766741/real 1775766741]  req@ffff9f14852d1880 x1862022114564480/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766766 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766741.003494:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1880 x1862022114564480/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766766 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766741.003502:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766741.003505:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766741.003509:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114564480:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766741.003516:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114564608:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766741.003536:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766741.003546:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766741/real 1775766741]  req@ffff9f14852d3100 x1862022114564608/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766766 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766741.003557:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3100 x1862022114564608/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766766 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766741.003566:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766741.003602:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766741.003607:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114564608:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766741.003613:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114564736:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766741.003636:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766741.003648:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766741/real 1775766741]  req@ffff9f14852d2300 x1862022114564736/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766766 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766741.003661:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2300 x1862022114564736/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766766 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766741.003670:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766741.003674:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766741.003677:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114564736:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775766741.258773:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766742.026667:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766742.282723:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766743.050647:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766743.306758:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766744.074622:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766744.330673:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766745.098708:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766745.354750:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766746.122664:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775766746.122683:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766746.122691:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775766746.122695:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766746.122706:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4123
00000100:00080000:3.0:1775766746.122719:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766746.122726:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775766746.122769:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747100 x1862022114564864/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766746.122799:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766746.122803:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775766746.122805:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766746.122809:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4123
00000100:00080000:3.0:1775766746.122813:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766746.122817:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766746.122828:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775766746.122842:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114564864:192.168.202.110@tcp:250:kworker.0
00000100:00100000:3.0:1775766746.122852:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745f80 x1862022114564992/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766746.122865:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766746.122869:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775766746.122871:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766746.122875:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4123
00000100:00080000:3.0:1775766746.122879:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766746.122881:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766746.122886:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766746.122896:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745180 x1862022114565120/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766746.122906:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766746.122909:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775766746.122910:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766746.122914:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4123
00000100:00080000:3.0:1775766746.122917:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000800:00000100:0.0:1775766746.122920:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775766746.122925:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766746.122929:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766746.122942:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745c00 x1862022114565248/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766746.122950:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766746.122953:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775766746.122955:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766746.122958:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4123
00000100:00080000:3.0:1775766746.122962:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766746.122966:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766746.122971:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775766746.122983:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766746/real 1775766746]  req@ffff9f1485747100 x1862022114564864/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766771 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766746.122990:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747800 x1862022114565376/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766746.123015:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1775766746.123018:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747100 x1862022114564864/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766771 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766746.123025:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775766746.123030:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766746.123041:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766746.123044:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4123
00000100:00080000:0.0:1775766746.123050:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:3.0:1775766746.123054:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1775766746.123055:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114564864:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775766746.123067:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766746.123073:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775766746.123086:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114564992:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766746.123112:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744000 x1862022114565504/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766746.123127:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766746.123131:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775766746.123133:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766746.123137:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4123
00000100:00080000:3.0:1775766746.123142:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000800:00000100:0.0:1775766746.123143:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775766746.123151:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766746.123157:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775766746.123165:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766746/real 1775766746]  req@ffff9f1485745f80 x1862022114564992/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766771 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766746.123177:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745500 x1862022114565632/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775766746.123185:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745f80 x1862022114564992/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766771 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766746.123207:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766746.123235:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775766746.123240:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766746.123241:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766746.123253:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00080000:3.0:1775766746.123255:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4123
00000100:00100000:0.0:1775766746.123261:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114564992:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775766746.123266:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1775766746.123271:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114565120:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775766746.123276:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766746.123287:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:0.0:1775766746.123313:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766746.123319:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744700 x1862022114565760/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775766746.123335:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766746/real 1775766746]  req@ffff9f1485745180 x1862022114565120/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766771 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766746.123349:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745180 x1862022114565120/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766771 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766746.123358:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766746.123362:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766746.123366:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114565120:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766746.123374:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114565248:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766746.123390:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766746.123425:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766746/real 1775766746]  req@ffff9f1485745c00 x1862022114565248/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766771 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766746.123435:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745c00 x1862022114565248/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766771 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766746.123443:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766746.123446:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766746.123448:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114565248:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775766746.123453:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114565376:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766746.123468:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766746.123475:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766746/real 1775766746]  req@ffff9f1485747800 x1862022114565376/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766771 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766746.123491:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747800 x1862022114565376/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766771 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766746.123498:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766746.123500:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766746.123503:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114565376:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775766746.123518:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114565504:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766746.123596:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766746.123608:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766746/real 1775766746]  req@ffff9f1485744000 x1862022114565504/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766771 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766746.123619:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744000 x1862022114565504/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766771 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766746.123626:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766746.123629:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766746.123631:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114565504:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766746.123636:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114565632:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766746.123651:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766746.123658:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766746/real 1775766746]  req@ffff9f1485745500 x1862022114565632/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766771 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766746.123667:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745500 x1862022114565632/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766771 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766746.123679:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766746.123682:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766746.123685:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114565632:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766746.123692:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114565760:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766746.123713:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766746.123727:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766746/real 1775766746]  req@ffff9f1485744700 x1862022114565760/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766771 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766746.123744:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744700 x1862022114565760/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766771 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766746.123756:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766746.123761:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766746.123765:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114565760:192.168.202.110@tcp:8:kworker.0
00010000:00010000:3.0:1775766746.378716:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766747.146677:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:0.0:1775766747.402686:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766748.170637:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766748.426681:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766749.194631:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766749.450674:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766750.218626:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766750.474686:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1775766751.242669:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766751.242677:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775766751.242681:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775766751.242689:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775766751.242690:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4128
00000100:00080000:3.0:1775766751.242702:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766751.242709:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775766751.242744:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746680 x1862022114565888/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766751.242768:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766751.242771:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775766751.242772:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766751.242775:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4128
00000100:00080000:3.0:1775766751.242777:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766751.242780:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766751.242788:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766751.242796:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744a80 x1862022114566016/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775766751.242801:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114565888:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775766751.242810:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766751.242813:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775766751.242814:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766751.242816:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4128
00000100:00080000:3.0:1775766751.242818:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766751.242821:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766751.242825:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766751.242832:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747480 x1862022114566144/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766751.242839:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766751.242842:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775766751.242843:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766751.242846:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4128
00000100:00080000:3.0:1775766751.242848:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766751.242850:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766751.242853:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766751.242859:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745880 x1862022114566272/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766751.242865:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766751.242867:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775766751.242869:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000800:00000100:0.0:1775766751.242871:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775766751.242875:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4128
00000100:00080000:3.0:1775766751.242877:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766751.242880:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766751.242884:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766751.242891:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746300 x1862022114566400/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766751.242897:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766751.242900:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775766751.242901:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766751.242904:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4128
00000100:00080000:3.0:1775766751.242907:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1775766751.242912:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766751/real 1775766751]  req@ffff9f1485746680 x1862022114565888/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766776 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766751.242912:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766751.242916:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766751.242927:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746a00 x1862022114566528/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766751.242943:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766751.242945:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775766751.242946:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775766751.242951:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746680 x1862022114565888/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766776 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766751.242951:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4128
00000100:00080000:3.0:1775766751.242954:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766751.242960:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766751.242968:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00080000:0.0:1775766751.242970:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766751.242975:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766751.242979:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744380 x1862022114566656/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775766751.242983:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114565888:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775766751.242994:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766751.242996:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775766751.242997:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766751.243001:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4128
00000100:00080000:3.0:1775766751.243004:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766751.243005:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766751.243009:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775766751.243029:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114566016:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766751.243041:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744e00 x1862022114566784/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1775766751.243065:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766751.243083:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766751/real 1775766751]  req@ffff9f1485744a80 x1862022114566016/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766776 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766751.243096:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744a80 x1862022114566016/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766776 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766751.243106:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766751.243124:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766751.243128:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114566016:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766751.243134:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114566144:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766751.243170:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766751.243176:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766751/real 1775766751]  req@ffff9f1485747480 x1862022114566144/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766776 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766751.243184:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747480 x1862022114566144/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766776 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766751.243193:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766751.243196:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766751.243198:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114566144:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766751.243201:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114566272:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766751.243211:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766751.243216:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766751/real 1775766751]  req@ffff9f1485745880 x1862022114566272/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766776 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766751.243223:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745880 x1862022114566272/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766776 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766751.243229:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766751.243231:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766751.243232:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114566272:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775766751.243235:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114566400:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766751.243244:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766751.243250:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766751/real 1775766751]  req@ffff9f1485746300 x1862022114566400/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766776 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766751.243257:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746300 x1862022114566400/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766776 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766751.243262:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766751.243264:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766751.243266:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114566400:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775766751.243270:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114566528:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766751.243280:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766751.243285:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766751/real 1775766751]  req@ffff9f1485746a00 x1862022114566528/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766776 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766751.243292:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746a00 x1862022114566528/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766776 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766751.243298:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766751.243299:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766751.243301:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114566528:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766751.243304:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114566656:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766751.243313:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766751.243318:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766751/real 1775766751]  req@ffff9f1485744380 x1862022114566656/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766776 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766751.243325:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744380 x1862022114566656/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766776 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766751.243331:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766751.243332:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766751.243334:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114566656:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766751.243346:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114566784:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766751.243355:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766751.243361:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766751/real 1775766751]  req@ffff9f1485744e00 x1862022114566784/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766776 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766751.243369:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744e00 x1862022114566784/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766776 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766751.243373:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766751.243375:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766751.243377:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114566784:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775766751.498738:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766752.266747:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766752.522840:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766753.290672:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775766753.546708:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766754.314722:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:0.0:1775766754.570704:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766755.338643:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766755.594742:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1775766756.362683:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1775766756.362684:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775766756.362699:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775766756.362705:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766756.362719:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4133
00000100:00080000:3.0:1775766756.362735:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766756.362744:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775766756.362795:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747b80 x1862022114566912/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766756.362830:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766756.362834:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775766756.362836:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766756.362850:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4133
00000100:00080000:3.0:1775766756.362857:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766756.362863:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766756.362875:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775766756.362885:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114566912:192.168.202.110@tcp:250:kworker.0
00000100:00100000:3.0:1775766756.362898:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744e00 x1862022114567040/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766756.362912:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766756.362916:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775766756.362918:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766756.362924:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4133
00000100:00080000:3.0:1775766756.362937:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766756.362942:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766756.362948:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766756.362962:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744380 x1862022114567168/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766756.362972:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766756.362976:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775766756.362979:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766756.362983:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4133
00000100:00080000:3.0:1775766756.362987:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766756.362991:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000800:00000100:0.0:1775766756.363000:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000008:00100000:3.0:1775766756.363004:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766756.363022:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746a00 x1862022114567296/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766756.363033:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766756.363036:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775766756.363039:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766756.363045:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4133
00000100:00080000:3.0:1775766756.363050:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766756.363055:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766756.363061:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775766756.363064:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766756/real 1775766756]  req@ffff9f1485747b80 x1862022114566912/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766781 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766756.363079:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746300 x1862022114567424/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775766756.363104:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747b80 x1862022114566912/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766781 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766756.363107:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766756.363125:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775766756.363131:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766756.363138:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766756.363140:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4133
00000100:00080000:3.0:1775766756.363147:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766756.363149:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:3.0:1775766756.363159:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775766756.363161:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114566912:192.168.202.110@tcp:250:kworker.0
00000008:00100000:3.0:1775766756.363174:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766756.363205:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745880 x1862022114567552/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766756.363217:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766756.363220:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00100000:0.0:1775766756.363224:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114567040:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775766756.363225:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766756.363243:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4133
00000100:00080000:3.0:1775766756.363250:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766756.363255:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766756.363261:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000800:00000100:0.0:1775766756.363281:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766756.363282:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747480 x1862022114567680/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766756.363300:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1775766756.363304:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766756/real 1775766756]  req@ffff9f1485744e00 x1862022114567040/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766781 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766756.363309:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775766756.363314:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766756.363326:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4133
00000100:00080000:3.0:1775766756.363337:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766756.363348:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775766756.363350:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744e00 x1862022114567040/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766781 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:3.0:1775766756.363358:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00080000:0.0:1775766756.363376:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00100000:3.0:1775766756.363383:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744a80 x1862022114567808/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775766756.363384:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766756.363397:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114567040:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766756.363407:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114567168:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766756.363465:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766756.363481:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766756/real 1775766756]  req@ffff9f1485744380 x1862022114567168/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766781 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766756.363496:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744380 x1862022114567168/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766781 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766756.363507:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766756.363511:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766756.363515:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114567168:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766756.363523:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114567296:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766756.363543:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766756.363580:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766756/real 1775766756]  req@ffff9f1485746a00 x1862022114567296/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766781 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766756.363604:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746a00 x1862022114567296/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766781 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766756.363614:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766756.363618:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766756.363622:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114567296:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775766756.363629:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114567424:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766756.363667:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766756.363679:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766756/real 1775766756]  req@ffff9f1485746300 x1862022114567424/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766781 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766756.363694:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746300 x1862022114567424/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766781 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766756.363704:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766756.363708:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766756.363712:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114567424:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775766756.363720:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114567552:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766756.363756:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766756.363766:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766756/real 1775766756]  req@ffff9f1485745880 x1862022114567552/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766781 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766756.363779:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745880 x1862022114567552/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766781 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766756.363795:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766756.363799:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766756.363803:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114567552:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766756.363828:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114567680:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766756.363849:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766756.363861:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766756/real 1775766756]  req@ffff9f1485747480 x1862022114567680/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766781 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766756.363874:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747480 x1862022114567680/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766781 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766756.363883:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766756.363886:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766756.363890:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114567680:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766756.363898:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114567808:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766756.363917:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766756.363931:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766756/real 1775766756]  req@ffff9f1485744a80 x1862022114567808/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766781 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766756.363943:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744a80 x1862022114567808/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766781 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766756.363952:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766756.363955:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766756.363959:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114567808:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775766756.618672:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766757.386641:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766757.642777:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766758.410635:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766758.666842:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766759.434682:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766759.690723:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766760.458666:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775766760.714775:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766761.482654:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:1.0:1775766761.482712:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766761.482725:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1775766761.482730:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766761.482760:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4138
00000100:00080000:1.0:1775766761.482773:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766761.482780:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775766761.482824:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdf80 x1862022114567936/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766761.482864:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766761.482869:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775766761.482872:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766761.482878:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4138
00000100:00080000:1.0:1775766761.482883:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766761.482888:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766761.482899:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775766761.482905:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114567936:192.168.202.110@tcp:250:kworker.0
00000100:00100000:1.0:1775766761.482919:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff800 x1862022114568064/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766761.482934:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766761.482937:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775766761.482939:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766761.482942:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4138
00000100:00080000:1.0:1775766761.482945:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766761.482950:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766761.482955:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775766761.482964:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fea00 x1862022114568192/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766761.482973:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766761.482977:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775766761.482983:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000800:00000100:0.0:1775766761.482984:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:1.0:1775766761.482991:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4138
00000100:00080000:1.0:1775766761.482995:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766761.482998:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766761.483003:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775766761.483012:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ffb80 x1862022114568320/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766761.483020:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766761.483023:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1775766761.483025:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766761.483029:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4138
00000100:00080000:1.0:1775766761.483033:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766761.483037:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766761.483041:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775766761.483051:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766761/real 1775766761]  req@ffff9f14ab4fdf80 x1862022114567936/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766786 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775766761.483053:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fce00 x1862022114568448/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766761.483132:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766761.483135:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775766761.483138:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766761.483150:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4138
00000100:00100000:0.0:1775766761.483151:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdf80 x1862022114567936/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766786 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766761.483161:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766761.483174:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00080000:0.0:1775766761.483178:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000008:00100000:1.0:1775766761.483179:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00080000:0.0:1775766761.483188:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766761.483194:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114567936:192.168.202.110@tcp:250:kworker.0
00000100:00100000:1.0:1775766761.483198:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fca80 x1862022114568576/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766761.483228:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766761.483230:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775766761.483232:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766761.483236:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4138
00000100:00080000:1.0:1775766761.483240:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1775766761.483245:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114568064:192.168.202.110@tcp:38:kworker.0
00000100:00080000:1.0:1775766761.483252:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766761.483257:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775766761.483269:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fed80 x1862022114568704/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766761.483279:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766761.483283:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775766761.483285:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000800:00000100:0.0:1775766761.483288:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:1.0:1775766761.483297:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4138
00000100:00080000:1.0:1775766761.483303:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766761.483308:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775766761.483312:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766761/real 1775766761]  req@ffff9f14ab4ff800 x1862022114568064/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766786 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:1.0:1775766761.483315:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775766761.483341:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdc00 x1862022114568832/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775766761.483345:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff800 x1862022114568064/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766786 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766761.483414:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766761.483420:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766761.483426:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114568064:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766761.483437:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114568192:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766761.483472:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766761.483482:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766761/real 1775766761]  req@ffff9f14ab4fea00 x1862022114568192/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766786 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766761.483491:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fea00 x1862022114568192/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766786 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766761.483497:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766761.483499:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766761.483502:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114568192:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766761.483507:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ffb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114568320:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766761.483521:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766761.483527:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766761/real 1775766761]  req@ffff9f14ab4ffb80 x1862022114568320/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766786 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766761.483535:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ffb80 x1862022114568320/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766786 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766761.483540:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766761.483542:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766761.483544:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ffb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114568320:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775766761.483548:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114568448:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766761.483582:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766761.483589:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766761/real 1775766761]  req@ffff9f14ab4fce00 x1862022114568448/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766786 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766761.483598:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fce00 x1862022114568448/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766786 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766761.483603:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766761.483606:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766761.483608:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114568448:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775766761.483612:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114568576:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766761.483623:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766761.483629:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766761/real 1775766761]  req@ffff9f14ab4fca80 x1862022114568576/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766786 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766761.483637:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fca80 x1862022114568576/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766786 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766761.483642:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766761.483643:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766761.483645:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114568576:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766761.483670:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114568704:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766761.483684:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766761.483694:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766761/real 1775766761]  req@ffff9f14ab4fed80 x1862022114568704/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766786 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766761.483707:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fed80 x1862022114568704/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766786 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766761.483717:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766761.483721:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766761.483725:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114568704:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766761.483732:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114568832:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766761.483751:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766761.483759:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766761/real 1775766761]  req@ffff9f14ab4fdc00 x1862022114568832/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766786 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766761.483767:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdc00 x1862022114568832/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766786 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766761.483773:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766761.483776:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766761.483778:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114568832:192.168.202.110@tcp:8:kworker.0
00010000:00010000:0.0:1775766761.738688:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766762.506710:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766762.762799:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766763.530636:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766763.786757:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766764.554670:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766764.810779:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766765.578660:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766765.834727:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766766.602666:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:1.0:1775766766.602669:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766766.602679:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1775766766.602684:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766766.602696:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4144
00000100:00080000:1.0:1775766766.602710:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766766.602719:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775766766.602763:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff100 x1862022114568960/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766766.602790:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766766.602793:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775766766.602795:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766766.602798:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4144
00000100:00080000:1.0:1775766766.602802:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766766.602805:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766766.602815:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775766766.602825:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114568960:192.168.202.110@tcp:250:kworker.0
00000100:00100000:1.0:1775766766.602829:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fe300 x1862022114569088/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766766.602853:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766766.602856:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775766766.602858:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766766.602862:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4144
00000100:00080000:1.0:1775766766.602865:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766766.602869:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766766.602873:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775766766.602885:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fd880 x1862022114569216/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766766.602892:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766766.602894:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775766766.602895:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766766.602898:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4144
00000100:00080000:1.0:1775766766.602927:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766766.602929:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766766.602933:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:0.0:1775766766.602940:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775766766.602952:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fc380 x1862022114569344/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766766.602959:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766766.602962:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1775766766.602963:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766766.602965:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4144
00000100:00080000:1.0:1775766766.602967:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766766.602970:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766766.602999:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775766766.603010:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdc00 x1862022114569472/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775766766.603017:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766766/real 1775766766]  req@ffff9f14ab4ff100 x1862022114568960/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766791 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766766.603020:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766766.603028:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775766766.603032:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766766.603040:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4144
00000100:00080000:1.0:1775766766.603045:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766766.603053:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766766.603058:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775766766.603066:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff100 x1862022114568960/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766791 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775766766.603069:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fed80 x1862022114569600/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766766.603085:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766766.603087:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775766766.603089:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766766.603093:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4144
00000100:00080000:1.0:1775766766.603100:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766766.603104:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775766766.603104:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00080000:0.0:1775766766.603113:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000008:00100000:1.0:1775766766.603113:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775766766.603120:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114568960:192.168.202.110@tcp:250:kworker.0
00000100:00100000:1.0:1775766766.603130:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fca80 x1862022114569728/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766766.603156:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766766.603160:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775766766.603162:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775766766.603165:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114569088:192.168.202.110@tcp:38:kworker.0
00000100:00080000:1.0:1775766766.603170:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4144
00000100:00080000:1.0:1775766766.603176:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766766.603181:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000800:00000100:0.0:1775766766.603189:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000008:00100000:1.0:1775766766.603189:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775766766.603200:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766766/real 1775766766]  req@ffff9f14ab4fe300 x1862022114569088/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766791 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775766766.603204:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fce00 x1862022114569856/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775766766.603243:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fe300 x1862022114569088/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766791 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766766.603251:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766766.603254:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766766.603258:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114569088:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766766.603263:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114569216:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766766.603281:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766766.603290:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766766/real 1775766766]  req@ffff9f14ab4fd880 x1862022114569216/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766791 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766766.603299:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fd880 x1862022114569216/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766791 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766766.603343:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766766.603347:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766766.603350:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114569216:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766766.603355:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114569344:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766766.603375:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766766.603382:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766766/real 1775766766]  req@ffff9f14ab4fc380 x1862022114569344/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766791 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766766.603390:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fc380 x1862022114569344/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766791 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766766.603397:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766766.603401:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766766.603403:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114569344:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775766766.603408:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114569472:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766766.603428:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766766.603441:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766766/real 1775766766]  req@ffff9f14ab4fdc00 x1862022114569472/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766791 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766766.603473:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdc00 x1862022114569472/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766791 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766766.603484:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766766.603490:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766766.603496:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114569472:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775766766.603505:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114569600:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766766.603526:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766766.603540:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766766/real 1775766766]  req@ffff9f14ab4fed80 x1862022114569600/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766791 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766766.603549:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fed80 x1862022114569600/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766791 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766766.603555:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766766.603558:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766766.603561:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114569600:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766766.603570:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114569728:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766766.603628:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766766.603637:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766766/real 1775766766]  req@ffff9f14ab4fca80 x1862022114569728/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766791 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766766.603644:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fca80 x1862022114569728/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766791 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766766.603649:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766766.603653:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766766.603656:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114569728:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766766.603675:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114569856:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766766.603688:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766766.603701:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766766/real 1775766766]  req@ffff9f14ab4fce00 x1862022114569856/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766791 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766766.603711:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fce00 x1862022114569856/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766791 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766766.603718:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766766.603720:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766766.603723:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114569856:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775766766.858774:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766767.626696:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775766767.882759:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766768.650701:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:0.0:1775766768.906654:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766769.674707:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766769.930735:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766770.698674:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766770.954760:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766771.722642:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:2.0:1775766771.722649:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766771.722658:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1775766771.722664:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766771.722676:0:8:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4149
00000100:00080000:2.0:1775766771.722688:0:8:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766771.722694:0:8:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775766771.722739:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3480 x1862022114569984/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766771.722765:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766771.722768:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775766771.722769:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766771.722773:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4149
00000100:00080000:2.0:1775766771.722775:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766771.722778:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766771.722789:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766771.722799:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1c00 x1862022114570112/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775766771.722806:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114569984:192.168.202.110@tcp:250:kworker.0
00000100:00080000:2.0:1775766771.722825:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766771.722828:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775766771.722829:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766771.722832:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4149
00000100:00080000:2.0:1775766771.722835:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766771.722838:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766771.722842:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766771.722851:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2300 x1862022114570240/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766771.722863:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766771.722865:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775766771.722867:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766771.722869:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4149
00000100:00080000:2.0:1775766771.722871:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766771.722874:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766771.722878:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775766771.722890:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3100 x1862022114570368/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1775766771.722899:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:2.0:1775766771.722913:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766771.722916:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1775766771.722918:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766771.722924:0:8:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4149
00000100:00080000:2.0:1775766771.722927:0:8:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766771.722930:0:8:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766771.722934:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775766771.722946:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1880 x1862022114570496/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775766771.722953:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766771/real 1775766771]  req@ffff9f14852d3480 x1862022114569984/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766796 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766771.722956:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766771.722975:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775766771.722981:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766771.722996:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4149
00000100:00080000:2.0:1775766771.723001:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1775766771.723004:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3480 x1862022114569984/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766796 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766771.723009:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766771.723023:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00080000:0.0:1775766771.723026:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766771.723034:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766771.723037:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114569984:192.168.202.110@tcp:250:kworker.0
00000100:00100000:2.0:1775766771.723042:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0e00 x1862022114570624/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766771.723051:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766771.723053:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775766771.723055:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766771.723058:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4149
00000100:00080000:2.0:1775766771.723060:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1775766771.723064:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114570112:192.168.202.110@tcp:38:kworker.0
00000100:00080000:2.0:1775766771.723068:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766771.723081:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766771.723094:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3b80 x1862022114570752/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1775766771.723102:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:2.0:1775766771.723103:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766771.723107:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775766771.723108:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766771.723112:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4149
00000100:00080000:2.0:1775766771.723116:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1775766771.723118:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766771/real 1775766771]  req@ffff9f14852d1c00 x1862022114570112/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766796 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766771.723124:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766771.723135:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775766771.723141:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1c00 x1862022114570112/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766796 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766771.723146:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0000 x1862022114570880/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775766771.723180:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766771.723193:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766771.723205:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114570112:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766771.723220:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114570240:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766771.723242:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766771.723250:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766771/real 1775766771]  req@ffff9f14852d2300 x1862022114570240/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766796 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766771.723279:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2300 x1862022114570240/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766796 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766771.723285:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766771.723288:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766771.723290:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114570240:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766771.723294:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114570368:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766771.723307:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766771.723313:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766771/real 1775766771]  req@ffff9f14852d3100 x1862022114570368/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766796 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766771.723325:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3100 x1862022114570368/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766796 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766771.723330:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766771.723332:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766771.723334:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114570368:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775766771.723338:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114570496:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766771.723347:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766771.723353:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766771/real 1775766771]  req@ffff9f14852d1880 x1862022114570496/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766796 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766771.723360:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1880 x1862022114570496/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766796 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766771.723365:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766771.723367:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766771.723368:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114570496:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775766771.723372:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114570624:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766771.723382:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766771.723387:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766771/real 1775766771]  req@ffff9f14852d0e00 x1862022114570624/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766796 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766771.723394:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0e00 x1862022114570624/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766796 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766771.723404:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766771.723406:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766771.723408:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114570624:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766771.723422:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114570752:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766771.723432:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766771.723438:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766771/real 1775766771]  req@ffff9f14852d3b80 x1862022114570752/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766796 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766771.723445:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3b80 x1862022114570752/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766796 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766771.723450:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766771.723452:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766771.723454:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114570752:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766771.723458:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114570880:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766771.723467:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766771.723473:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766771/real 1775766771]  req@ffff9f14852d0000 x1862022114570880/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766796 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766771.723480:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0000 x1862022114570880/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766796 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766771.723485:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766771.723487:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766771.723490:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114570880:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775766771.978720:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766772.746679:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766773.002827:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766773.770697:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766774.026751:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766774.794649:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775766775.050778:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766775.818673:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:0.0:1775766776.074646:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766776.842652:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775766776.842709:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766776.842718:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775766776.842721:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766776.842729:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4154
00000100:00080000:3.0:1775766776.842739:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766776.842744:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775766776.842780:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746680 x1862022114571008/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766776.842804:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766776.842807:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775766776.842808:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766776.842812:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4154
00000100:00080000:3.0:1775766776.842814:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766776.842817:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766776.842825:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766776.842834:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744700 x1862022114571136/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775766776.842844:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114571008:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775766776.842846:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766776.842856:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775766776.842859:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766776.842863:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4154
00000100:00080000:3.0:1775766776.842865:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766776.842868:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766776.842871:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766776.842882:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745500 x1862022114571264/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766776.842887:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766776.842889:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775766776.842891:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766776.842894:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4154
00000100:00080000:3.0:1775766776.842897:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766776.842901:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766776.842904:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766776.842909:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744000 x1862022114571392/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766776.842923:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000800:00000100:0.0:1775766776.842931:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775766776.842932:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775766776.842934:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766776.842938:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4154
00000100:00080000:3.0:1775766776.842942:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766776.842945:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766776.842950:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766776.842960:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747800 x1862022114571520/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766776.842968:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766776.842970:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775766776.842971:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766776.842974:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4154
00000100:00080000:3.0:1775766776.842979:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1775766776.842983:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766776/real 1775766776]  req@ffff9f1485746680 x1862022114571008/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766801 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766776.842991:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766776.843006:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766776.843020:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745c00 x1862022114571648/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775766776.843021:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746680 x1862022114571008/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766801 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766776.843083:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766776.843083:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766776.843095:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:0.0:1775766776.843100:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:3.0:1775766776.843102:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775766776.843110:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114571008:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775766776.843117:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4154
00000100:00080000:3.0:1775766776.843122:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766776.843126:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766776.843132:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775766776.843138:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114571136:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766776.843148:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745180 x1862022114571776/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1775766776.843160:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766776.843172:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766776/real 1775766776]  req@ffff9f1485744700 x1862022114571136/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766801 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766776.843176:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766776.843186:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00100000:0.0:1775766776.843189:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744700 x1862022114571136/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766801 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766776.843189:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766776.843211:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4154
00000100:00080000:3.0:1775766776.843220:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766776.843229:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766776.843240:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00080000:0.0:1775766776.843242:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766776.843250:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766776.843255:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114571136:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766776.843255:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745f80 x1862022114571904/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775766776.843275:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114571264:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766776.843300:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766776.843308:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766776/real 1775766776]  req@ffff9f1485745500 x1862022114571264/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766801 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766776.843317:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745500 x1862022114571264/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766801 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766776.843322:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766776.843325:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766776.843327:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114571264:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766776.843331:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114571392:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766776.843341:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766776.843347:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766776/real 1775766776]  req@ffff9f1485744000 x1862022114571392/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766801 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766776.843354:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744000 x1862022114571392/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766801 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766776.843362:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766776.843365:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766776.843368:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114571392:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775766776.843374:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114571520:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766776.843395:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766776.843406:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766776/real 1775766776]  req@ffff9f1485747800 x1862022114571520/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766801 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766776.843420:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747800 x1862022114571520/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766801 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766776.843426:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766776.843429:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766776.843431:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114571520:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775766776.843436:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114571648:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766776.843451:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766776.843459:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766776/real 1775766776]  req@ffff9f1485745c00 x1862022114571648/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766801 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766776.843467:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745c00 x1862022114571648/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766801 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766776.843472:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766776.843474:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766776.843476:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114571648:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766776.843491:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114571776:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766776.843501:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766776.843507:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766776/real 1775766776]  req@ffff9f1485745180 x1862022114571776/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766801 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766776.843519:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745180 x1862022114571776/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766801 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766776.843524:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766776.843526:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766776.843527:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114571776:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766776.843531:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114571904:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766776.843541:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766776.843546:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766776/real 1775766776]  req@ffff9f1485745f80 x1862022114571904/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766801 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766776.843574:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745f80 x1862022114571904/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766801 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766776.843579:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766776.843581:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766776.843583:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114571904:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775766777.098777:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766777.866667:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766778.122798:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766778.890673:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766779.146763:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766779.914654:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766780.170795:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766780.938797:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766781.194826:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766781.962652:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:1.0:1775766781.962706:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766781.962715:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1775766781.962720:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766781.962730:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4159
00000100:00080000:1.0:1775766781.962744:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766781.962751:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775766781.962802:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ffb80 x1862022114572032/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766781.962833:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766781.962837:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775766781.962840:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766781.962845:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4159
00000100:00080000:1.0:1775766781.962873:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766781.962878:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766781.962890:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775766781.962902:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ffb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114572032:192.168.202.110@tcp:250:kworker.0
00000100:00100000:1.0:1775766781.962904:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fea00 x1862022114572160/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766781.962915:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766781.962919:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775766781.962921:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766781.962924:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4159
00000100:00080000:1.0:1775766781.962929:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766781.962932:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766781.962937:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775766781.962946:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff800 x1862022114572288/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766781.962955:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766781.962958:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775766781.962959:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766781.962963:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4159
00000100:00080000:1.0:1775766781.962967:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766781.962971:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000800:00000100:0.0:1775766781.962973:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000008:00100000:1.0:1775766781.962976:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775766781.962985:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdf80 x1862022114572416/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766781.962993:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766781.962997:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1775766781.962998:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766781.963002:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4159
00000100:00080000:1.0:1775766781.963004:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766781.963008:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766781.963012:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775766781.963019:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766781/real 1775766781]  req@ffff9f14ab4ffb80 x1862022114572032/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766806 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775766781.963022:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fce00 x1862022114572544/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766781.963040:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766781.963044:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775766781.963046:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766781.963050:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4159
00000100:00080000:1.0:1775766781.963054:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1775766781.963057:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ffb80 x1862022114572032/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766806 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766781.963058:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766781.963063:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00080000:0.0:1775766781.963069:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766781.963073:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775766781.963073:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fca80 x1862022114572672/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775766781.963079:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ffb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114572032:192.168.202.110@tcp:250:kworker.0
00000100:00080000:1.0:1775766781.963083:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766781.963087:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775766781.963088:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766781.963092:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4159
00000100:00080000:1.0:1775766781.963096:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766781.963100:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775766781.963104:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114572160:192.168.202.110@tcp:38:kworker.0
00000008:00100000:1.0:1775766781.963105:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775766781.963114:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fed80 x1862022114572800/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766781.963125:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000800:00000100:0.0:1775766781.963128:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:1.0:1775766781.963129:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775766781.963131:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766781.963135:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4159
00000100:00100000:0.0:1775766781.963140:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766781/real 1775766781]  req@ffff9f14ab4fea00 x1862022114572160/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766806 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766781.963140:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766781.963144:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766781.963165:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775766781.963169:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fea00 x1862022114572160/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766806 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775766781.963176:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdc00 x1862022114572928/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775766781.963178:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766781.963182:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766781.963186:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114572160:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766781.963192:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114572288:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766781.963214:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766781.963226:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766781/real 1775766781]  req@ffff9f14ab4ff800 x1862022114572288/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766806 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766781.963238:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff800 x1862022114572288/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766806 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766781.963249:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766781.963253:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766781.963256:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114572288:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766781.963264:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114572416:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766781.963288:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766781.963300:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766781/real 1775766781]  req@ffff9f14ab4fdf80 x1862022114572416/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766806 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766781.963314:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdf80 x1862022114572416/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766806 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766781.963324:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766781.963326:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766781.963329:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114572416:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775766781.963334:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114572544:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766781.963350:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766781.963358:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766781/real 1775766781]  req@ffff9f14ab4fce00 x1862022114572544/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766806 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766781.963366:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fce00 x1862022114572544/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766806 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766781.963372:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766781.963374:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766781.963377:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114572544:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775766781.963381:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114572672:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766781.963392:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766781.963398:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766781/real 1775766781]  req@ffff9f14ab4fca80 x1862022114572672/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766806 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766781.963405:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fca80 x1862022114572672/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766806 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766781.963411:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766781.963412:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766781.963414:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114572672:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766781.963429:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114572800:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766781.963439:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766781.963446:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766781/real 1775766781]  req@ffff9f14ab4fed80 x1862022114572800/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766806 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766781.963454:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fed80 x1862022114572800/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766806 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766781.963467:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766781.963470:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766781.963472:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114572800:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766781.963476:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114572928:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766781.963488:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766781.963495:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766781/real 1775766781]  req@ffff9f14ab4fdc00 x1862022114572928/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766806 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766781.963503:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdc00 x1862022114572928/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766806 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766781.963509:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766781.963511:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766781.963513:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114572928:192.168.202.110@tcp:8:kworker.0
00010000:00010000:3.0:1775766782.218780:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766782.986624:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:0.0:1775766783.242705:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766784.010667:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766784.266749:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766785.034646:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766785.290763:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766786.058648:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766786.314761:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766787.082631:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775766787.082735:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766787.082741:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775766787.082745:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766787.082754:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4164
00000100:00080000:3.0:1775766787.082765:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766787.082771:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775766787.082808:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747100 x1862022114573056/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766787.082853:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766787.082857:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775766787.082859:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766787.082863:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4164
00000100:00080000:3.0:1775766787.082867:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766787.082871:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766787.082881:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766787.082895:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745f80 x1862022114573184/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766787.082905:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766787.082908:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775766787.082910:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766787.082913:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4164
00000100:00080000:3.0:1775766787.082931:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766787.082935:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766787.082939:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766787.082956:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745180 x1862022114573312/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766787.082966:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766787.082968:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775766787.082970:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766787.082975:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4164
00000100:00100000:0.0:1775766787.082977:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114573056:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775766787.082978:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766787.083003:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766787.083009:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766787.083020:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745c00 x1862022114573440/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766787.083029:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766787.083033:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775766787.083036:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766787.083040:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4164
00000100:00080000:3.0:1775766787.083044:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766787.083046:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766787.083053:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766787.083067:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747800 x1862022114573568/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1775766787.083080:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775766787.083155:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766787.083160:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775766787.083163:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766787.083169:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4164
00000100:00100000:0.0:1775766787.083172:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766787/real 1775766787]  req@ffff9f1485747100 x1862022114573056/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766812 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766787.083178:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766787.083187:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766787.083193:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775766787.083198:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747100 x1862022114573056/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766812 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766787.083209:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00100000:3.0:1775766787.083209:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744000 x1862022114573696/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775766787.083218:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:3.0:1775766787.083223:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766787.083225:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775766787.083229:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775766787.083233:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114573056:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775766787.083236:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4164
00000100:00080000:3.0:1775766787.083240:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766787.083247:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775766787.083252:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114573184:192.168.202.110@tcp:38:kworker.0
00000008:00100000:3.0:1775766787.083254:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766787.083267:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745500 x1862022114573824/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766787.083274:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766787.083277:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000800:00000100:0.0:1775766787.083281:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775766787.083282:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766787.083290:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4164
00000100:00080000:3.0:1775766787.083293:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1775766787.083299:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766787/real 1775766787]  req@ffff9f1485745f80 x1862022114573184/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766812 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766787.083299:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766787.083303:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775766787.083322:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745f80 x1862022114573184/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766812 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766787.083323:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744700 x1862022114573952/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775766787.083345:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766787.083351:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766787.083356:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114573184:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766787.083379:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114573312:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766787.083392:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766787.083398:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766787/real 1775766787]  req@ffff9f1485745180 x1862022114573312/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766812 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766787.083407:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745180 x1862022114573312/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766812 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766787.083412:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766787.083414:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766787.083417:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114573312:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766787.083421:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114573440:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766787.083432:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766787.083438:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766787/real 1775766787]  req@ffff9f1485745c00 x1862022114573440/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766812 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766787.083445:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745c00 x1862022114573440/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766812 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766787.083451:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766787.083453:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766787.083455:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114573440:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775766787.083459:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114573568:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766787.083469:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766787.083475:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766787/real 1775766787]  req@ffff9f1485747800 x1862022114573568/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766812 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766787.083482:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747800 x1862022114573568/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766812 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766787.083488:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766787.083489:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766787.083491:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114573568:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775766787.083495:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114573696:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766787.083505:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766787.083516:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766787/real 1775766787]  req@ffff9f1485744000 x1862022114573696/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766812 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766787.083523:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744000 x1862022114573696/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766812 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766787.083529:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766787.083531:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766787.083533:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114573696:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766787.083536:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114573824:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766787.083548:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766787.083597:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766787/real 1775766787]  req@ffff9f1485745500 x1862022114573824/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766812 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766787.083612:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745500 x1862022114573824/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766812 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766787.083619:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766787.083622:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766787.083625:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114573824:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766787.083630:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114573952:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766787.083647:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766787.083655:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766787/real 1775766787]  req@ffff9f1485744700 x1862022114573952/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766812 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766787.083665:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744700 x1862022114573952/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766812 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766787.083671:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766787.083673:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766787.083675:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114573952:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775766787.338765:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766788.106686:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766788.362787:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766789.130655:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775766789.386740:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766790.154644:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:0.0:1775766790.410646:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766791.178715:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766791.434687:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1775766792.202658:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766792.202667:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00100000:2.0:1775766792.202669:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775766792.202671:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766792.202701:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4169
00000100:00080000:3.0:1775766792.202715:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766792.202722:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775766792.202764:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746680 x1862022114574080/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766792.202819:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766792.202822:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775766792.202824:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766792.202828:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4169
00000100:00080000:3.0:1775766792.202831:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766792.202835:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766792.202844:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766792.202855:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744a80 x1862022114574208/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766792.202864:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766792.202867:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775766792.202869:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775766792.202873:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114574080:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775766792.202875:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4169
00000100:00080000:3.0:1775766792.202880:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766792.202885:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766792.202888:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766792.202897:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747480 x1862022114574336/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766792.202904:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766792.202913:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775766792.202915:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766792.202918:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4169
00000100:00080000:3.0:1775766792.202922:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766792.202925:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766792.202928:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766792.202937:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745880 x1862022114574464/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766792.202944:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766792.202947:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000800:00000100:0.0:1775766792.202949:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775766792.202950:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766792.202957:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4169
00000100:00080000:3.0:1775766792.202959:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766792.202962:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766792.202966:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766792.202975:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746300 x1862022114574592/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766792.202981:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766792.202983:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775766792.202985:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766792.202988:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4169
00000100:00080000:3.0:1775766792.202991:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766792.202994:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775766792.202999:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766792/real 1775766792]  req@ffff9f1485746680 x1862022114574080/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766817 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:3.0:1775766792.203000:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766792.203015:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746a00 x1862022114574720/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775766792.203023:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746680 x1862022114574080/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766817 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766792.203023:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766792.203029:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:0.0:1775766792.203056:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766792.203058:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766792.203066:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:3.0:1775766792.203069:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4169
00000100:00100000:0.0:1775766792.203076:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114574080:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775766792.203077:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766792.203087:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766792.203094:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775766792.203109:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114574208:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766792.203112:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744380 x1862022114574848/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766792.203129:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766792.203133:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775766792.203136:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766792.203141:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4169
00000100:00080000:3.0:1775766792.203146:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000800:00000100:0.0:1775766792.203147:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775766792.203153:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766792.203159:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775766792.203170:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766792/real 1775766792]  req@ffff9f1485744a80 x1862022114574208/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766817 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766792.203172:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744e00 x1862022114574976/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775766792.203219:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744a80 x1862022114574208/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766817 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766792.203227:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766792.203231:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766792.203234:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114574208:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766792.203239:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114574336:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766792.203257:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766792.203264:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766792/real 1775766792]  req@ffff9f1485747480 x1862022114574336/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766817 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766792.203272:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747480 x1862022114574336/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766817 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766792.203289:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766792.203291:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766792.203293:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114574336:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766792.203298:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114574464:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766792.203309:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766792.203315:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766792/real 1775766792]  req@ffff9f1485745880 x1862022114574464/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766817 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766792.203323:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745880 x1862022114574464/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766817 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766792.203329:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766792.203331:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766792.203333:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114574464:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775766792.203337:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114574592:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766792.203347:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766792.203353:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766792/real 1775766792]  req@ffff9f1485746300 x1862022114574592/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766817 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766792.203361:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746300 x1862022114574592/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766817 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766792.203369:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766792.203373:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766792.203376:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114574592:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775766792.203383:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114574720:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766792.203404:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766792.203416:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766792/real 1775766792]  req@ffff9f1485746a00 x1862022114574720/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766817 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766792.203432:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746a00 x1862022114574720/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766817 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766792.203440:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766792.203443:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766792.203445:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114574720:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766792.203461:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114574848:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766792.203478:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766792.203487:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766792/real 1775766792]  req@ffff9f1485744380 x1862022114574848/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766817 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766792.203501:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744380 x1862022114574848/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766817 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766792.203511:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766792.203515:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766792.203519:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114574848:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766792.203526:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114574976:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766792.203596:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766792.203607:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766792/real 1775766792]  req@ffff9f1485744e00 x1862022114574976/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766817 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766792.203616:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744e00 x1862022114574976/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766817 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766792.203628:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766792.203630:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766792.203633:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114574976:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775766792.458791:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766793.226669:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775766793.482674:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766794.250633:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766794.506706:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766795.274689:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766795.530791:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766796.298633:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775766796.554753:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766797.322719:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:0.0:1775766797.322754:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766797.322769:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1775766797.322776:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766797.322793:0:8:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4174
00000100:00080000:0.0:1775766797.322810:0:8:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766797.322819:0:8:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775766797.322870:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d2300 x1862022114575104/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775766797.322911:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766797.322916:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775766797.322919:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766797.322924:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4174
00000100:00080000:0.0:1775766797.322929:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766797.322933:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775766797.322948:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775766797.322974:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3100 x1862022114575232/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775766797.322980:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114575104:192.168.202.110@tcp:250:kworker.0
00000100:00080000:0.0:1775766797.323038:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766797.323046:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:0.0:1775766797.323052:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766797.323062:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4174
00000100:00080000:0.0:1775766797.323071:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766797.323082:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775766797.323091:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775766797.323113:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3b80 x1862022114575360/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775766797.323127:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766797.323144:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775766797.323148:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766797.323154:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4174
00000100:00080000:0.0:1775766797.323162:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766797.323170:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775766797.323178:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775766797.323187:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114575232:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766797.323198:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d2a00 x1862022114575488/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775766797.323213:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766797.323218:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1775766797.323221:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766797.323226:0:8:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4174
00000100:00080000:0.0:1775766797.323234:0:8:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766797.323249:0:8:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775766797.323256:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775766797.323258:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114575360:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766797.323286:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0a80 x1862022114575616/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775766797.323301:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766797.323306:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00100000:2.0:1775766797.323311:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114575488:192.168.202.110@tcp:8:kworker.0
00000100:00080000:0.0:1775766797.323313:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766797.323329:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4174
00000100:00080000:0.0:1775766797.323336:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766797.323341:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775766797.323349:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766797.323367:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114575616:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775766797.323372:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3800 x1862022114575744/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775766797.323391:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766797.323396:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:0.0:1775766797.323400:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766797.323407:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4174
00000100:00080000:0.0:1775766797.323414:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:2.0:1775766797.323418:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114575744:192.168.202.110@tcp:38:kworker.0
00000100:00080000:0.0:1775766797.323425:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775766797.323447:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775766797.323467:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d1500 x1862022114575872/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775766797.323485:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766797.323490:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775766797.323492:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766797.323498:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4174
00000100:00080000:0.0:1775766797.323504:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766797.323510:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775766797.323517:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775766797.323518:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114575872:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766797.323538:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0000 x1862022114576000/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775766797.323601:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114576000:192.168.202.110@tcp:8:kworker.0
00010000:00010000:0.0:1775766797.578729:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766798.346688:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766798.602787:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766799.370642:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766799.626794:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766800.394667:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000400:00000100:2.0:1775766800.395316:0:2392:0:(lib-socket.c:454:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.202.110:988
00000400:00000100:2.0:1775766800.395378:0:2392:0:(acceptor.c:111:lnet_connect_console_error()) Connection to 192.168.202.110@tcp at host 192.168.202.110 was unreachable: the network or that node may be down, or Lustre may be misconfigured.
00000800:00000100:2.0:1775766800.395430:0:2392:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:2.0:1775766800.395499:0:2392:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:2.0:1775766800.395517:0:2392:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:2.0:1775766800.395546:0:2392:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:2.0:1775766800.395562:0:2392:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000100:00100000:0.0:1775766800.395625:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766797/real 1775766800]  req@ffff9f14a04d2a00 x1862022114575488/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766822 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000800:00000100:2.0:1775766800.395639:0:2392:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:2.0:1775766800.395653:0:2392:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:2.0:1775766800.395675:0:2392:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000100:00100000:0.0:1775766800.395712:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d2a00 x1862022114575488/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766822 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766800.395734:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766800.395743:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766800.395750:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114575488:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775766800.395762:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766797/real 1775766800]  req@ffff9f14a04d0a80 x1862022114575616/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766822 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766800.395784:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0a80 x1862022114575616/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766822 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766800.395797:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766800.395803:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766800.395809:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114575616:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775766800.395820:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766797/real 1775766800]  req@ffff9f14a04d3100 x1862022114575232/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766822 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766800.395860:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3100 x1862022114575232/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766822 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766800.395873:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766800.395879:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766800.395884:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114575232:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766800.395894:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766797/real 1775766800]  req@ffff9f14a04d3b80 x1862022114575360/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766822 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766800.395912:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3b80 x1862022114575360/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766822 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766800.395923:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766800.395928:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766800.395932:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114575360:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766800.395942:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766797/real 1775766800]  req@ffff9f14a04d2300 x1862022114575104/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766822 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766800.395987:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d2300 x1862022114575104/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766822 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766800.396000:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766800.396006:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766800.396011:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114575104:192.168.202.110@tcp:250:kworker.0
00000100:00100000:0.0:1775766800.396063:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766797/real 1775766800]  req@ffff9f14a04d0000 x1862022114576000/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766822 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766800.396084:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0000 x1862022114576000/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766822 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766800.396095:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766800.396101:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766800.396106:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114576000:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775766800.396115:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766797/real 1775766800]  req@ffff9f14a04d1500 x1862022114575872/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766822 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766800.396132:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1500 x1862022114575872/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766822 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766800.396142:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766800.396147:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766800.396151:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114575872:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766800.396161:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766797/real 1775766800]  req@ffff9f14a04d3800 x1862022114575744/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766822 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766800.396178:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3800 x1862022114575744/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766822 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766800.396190:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766800.396195:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766800.396200:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114575744:192.168.202.110@tcp:38:kworker.0
00010000:00010000:1.0:1775766800.650865:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766801.418767:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766801.674820:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766802.442615:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775766802.442709:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766802.442715:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775766802.442718:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766802.442727:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4179
00000100:00080000:3.0:1775766802.442737:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766802.442744:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775766802.442776:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747b80 x1862022114576128/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766802.442800:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766802.442804:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775766802.442806:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766802.442811:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4179
00000100:00080000:3.0:1775766802.442816:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766802.442821:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766802.442832:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766802.442852:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744e00 x1862022114576256/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766802.442865:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766802.442870:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00100000:0.0:1775766802.442872:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114576128:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775766802.442872:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766802.442901:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4179
00000100:00080000:3.0:1775766802.442908:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766802.442913:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766802.442921:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766802.442949:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744380 x1862022114576384/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766802.442960:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766802.442963:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775766802.442965:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766802.442970:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4179
00000100:00080000:3.0:1775766802.442975:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766802.442980:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766802.442988:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:0.0:1775766802.442990:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766802.443012:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746a00 x1862022114576512/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766802.443026:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766802.443029:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775766802.443031:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766802.443037:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4179
00000100:00080000:3.0:1775766802.443043:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766802.443050:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766802.443057:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775766802.443070:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766802/real 1775766802]  req@ffff9f1485747b80 x1862022114576128/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766827 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766802.443077:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746300 x1862022114576640/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766802.443092:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766802.443116:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775766802.443121:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766802.443130:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4179
00000100:00080000:3.0:1775766802.443138:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1775766802.443144:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747b80 x1862022114576128/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766827 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766802.443150:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766802.443165:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00080000:0.0:1775766802.443176:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766802.443186:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766802.443194:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745880 x1862022114576768/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775766802.443200:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114576128:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775766802.443227:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766802.443232:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775766802.443236:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766802.443243:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4179
00000100:00080000:3.0:1775766802.443249:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1775766802.443259:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114576256:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775766802.443260:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766802.443280:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766802.443303:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747480 x1862022114576896/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1775766802.443319:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766802.443342:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766802/real 1775766802]  req@ffff9f1485744e00 x1862022114576256/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766827 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766802.443361:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744e00 x1862022114576256/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766827 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766802.443373:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766802.443378:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766802.443383:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114576256:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766802.443392:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114576384:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766802.443421:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766802.443435:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766802/real 1775766802]  req@ffff9f1485744380 x1862022114576384/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766827 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766802.443462:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744380 x1862022114576384/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766827 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766802.443473:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766802.443477:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766802.443480:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114576384:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766802.443489:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114576512:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766802.443517:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766802.443540:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766802/real 1775766802]  req@ffff9f1485746a00 x1862022114576512/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766827 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766802.443594:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746a00 x1862022114576512/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766827 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766802.443607:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766802.443613:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766802.443618:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114576512:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775766802.443628:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114576640:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766802.443659:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766802.443674:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766802/real 1775766802]  req@ffff9f1485746300 x1862022114576640/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766827 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766802.443689:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746300 x1862022114576640/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766827 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766802.443707:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766802.443711:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766802.443715:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114576640:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775766802.443724:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114576768:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766802.443751:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766802.443765:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766802/real 1775766802]  req@ffff9f1485745880 x1862022114576768/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766827 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766802.443780:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745880 x1862022114576768/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766827 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766802.443790:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766802.443794:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766802.443798:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114576768:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766802.443825:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114576896:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766802.443857:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766802.443879:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766802/real 1775766802]  req@ffff9f1485747480 x1862022114576896/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766827 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766802.443897:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747480 x1862022114576896/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766827 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766802.443907:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766802.443911:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766802.443917:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114576896:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775766802.443955:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766802.443962:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775766802.443966:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766802.443972:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4179
00000100:00080000:3.0:1775766802.443980:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766802.443986:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766802.443995:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766802.444017:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744a80 x1862022114577024/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775766802.444129:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114577024:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766802.444168:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766802.444194:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766802/real 1775766802]  req@ffff9f1485744a80 x1862022114577024/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766827 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766802.444213:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744a80 x1862022114577024/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766827 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766802.444228:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766802.444233:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766802.444238:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114577024:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775766802.698808:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766803.466684:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775766803.722774:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766804.490645:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:0.0:1775766804.746750:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766805.514655:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766805.770894:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766806.538642:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766806.794749:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766807.562714:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:1.0:1775766807.562731:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766807.562739:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1775766807.562744:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766807.562756:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4185
00000100:00080000:1.0:1775766807.562770:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766807.562777:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775766807.562820:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fc380 x1862022114577152/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766807.562874:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766807.562911:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775766807.562913:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766807.562918:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4185
00000100:00080000:1.0:1775766807.562923:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766807.562927:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766807.562939:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775766807.562957:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fd880 x1862022114577280/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766807.562979:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1775766807.562983:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114577152:192.168.202.110@tcp:250:kworker.0
00000100:00080000:1.0:1775766807.562984:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775766807.562988:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766807.563000:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4185
00000100:00080000:1.0:1775766807.563004:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766807.563007:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766807.563011:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775766807.563024:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fe300 x1862022114577408/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766807.563034:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766807.563039:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775766807.563041:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766807.563055:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4185
00000100:00080000:1.0:1775766807.563072:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766807.563077:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766807.563082:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:0.0:1775766807.563090:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775766807.563098:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff100 x1862022114577536/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766807.563111:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766807.563115:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1775766807.563117:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766807.563122:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4185
00000100:00080000:1.0:1775766807.563126:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766807.563130:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766807.563136:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775766807.563150:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdc00 x1862022114577664/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766807.563160:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1775766807.563163:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766807/real 1775766807]  req@ffff9f14ab4fc380 x1862022114577152/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766832 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766807.563164:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775766807.563169:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766807.563180:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4185
00000100:00080000:1.0:1775766807.563188:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766807.563196:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766807.563203:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775766807.563215:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fed80 x1862022114577792/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775766807.563221:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fc380 x1862022114577152/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766832 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766807.563238:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766807.563258:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775766807.563259:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775766807.563264:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766807.563272:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:1.0:1775766807.563273:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4185
00000100:00080000:1.0:1775766807.563284:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766807.563296:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775766807.563302:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114577152:192.168.202.110@tcp:250:kworker.0
00000008:00100000:1.0:1775766807.563304:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775766807.563323:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fca80 x1862022114577920/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766807.563340:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766807.563344:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775766807.563347:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775766807.563353:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114577280:192.168.202.110@tcp:38:kworker.0
00000100:00080000:1.0:1775766807.563356:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4185
00000100:00080000:1.0:1775766807.563365:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766807.563373:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766807.563380:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775766807.563393:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fce00 x1862022114578048/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1775766807.563424:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766807.563449:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766807/real 1775766807]  req@ffff9f14ab4fd880 x1862022114577280/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766832 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766807.563471:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fd880 x1862022114577280/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766832 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766807.563485:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766807.563491:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766807.563496:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114577280:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766807.563507:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114577408:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766807.563541:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766807.563604:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766807/real 1775766807]  req@ffff9f14ab4fe300 x1862022114577408/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766832 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766807.563636:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fe300 x1862022114577408/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766832 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766807.563649:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766807.563654:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766807.563660:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114577408:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766807.563670:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114577536:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766807.563701:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766807.563716:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766807/real 1775766807]  req@ffff9f14ab4ff100 x1862022114577536/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766832 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766807.563746:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff100 x1862022114577536/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766832 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766807.563759:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766807.563764:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766807.563770:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114577536:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775766807.563779:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114577664:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766807.563809:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766807.563825:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766807/real 1775766807]  req@ffff9f14ab4fdc00 x1862022114577664/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766832 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766807.563844:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdc00 x1862022114577664/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766832 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766807.563875:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766807.563912:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766807.563934:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114577664:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775766807.563943:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114577792:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766807.563973:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766807.563988:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766807/real 1775766807]  req@ffff9f14ab4fed80 x1862022114577792/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766832 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766807.564007:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fed80 x1862022114577792/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766832 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766807.564030:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766807.564035:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766807.564040:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114577792:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766807.564050:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114577920:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766807.564083:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766807.564100:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766807/real 1775766807]  req@ffff9f14ab4fca80 x1862022114577920/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766832 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766807.564119:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fca80 x1862022114577920/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766832 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766807.564129:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766807.564132:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766807.564135:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114577920:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766807.564180:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114578048:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766807.564220:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766807.564243:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766807/real 1775766807]  req@ffff9f14ab4fce00 x1862022114578048/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766832 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766807.564266:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fce00 x1862022114578048/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766832 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766807.564282:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766807.564288:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766807.564295:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114578048:192.168.202.110@tcp:8:kworker.0
00010000:00010000:1.0:1775766807.818823:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766808.586625:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766808.842730:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766809.610620:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766809.866713:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766810.634634:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775766810.890783:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766811.658706:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:0.0:1775766811.914721:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1775766812.682730:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:2.0:1775766812.682733:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775766812.682740:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775766812.682748:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766812.682758:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4190
00000100:00080000:3.0:1775766812.682769:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766812.682775:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775766812.682813:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746680 x1862022114578176/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766812.682883:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766812.682887:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775766812.682890:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766812.682894:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4190
00000100:00080000:3.0:1775766812.682899:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766812.682904:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766812.682924:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775766812.682925:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114578176:192.168.202.110@tcp:250:kworker.0
00000100:00100000:3.0:1775766812.682940:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744700 x1862022114578304/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766812.682950:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766812.682952:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775766812.682954:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766812.682957:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4190
00000100:00080000:3.0:1775766812.682961:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766812.682963:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766812.682967:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766812.682973:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745500 x1862022114578432/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766812.682980:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766812.682982:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775766812.682983:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766812.682986:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4190
00000100:00080000:3.0:1775766812.682989:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766812.682991:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766812.682994:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:0.0:1775766812.682999:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766812.683005:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744000 x1862022114578560/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766812.683011:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766812.683014:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775766812.683015:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766812.683018:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4190
00000100:00080000:3.0:1775766812.683021:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766812.683023:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766812.683026:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766812.683033:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747800 x1862022114578688/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766812.683039:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766812.683041:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775766812.683044:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775766812.683045:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766812/real 1775766812]  req@ffff9f1485746680 x1862022114578176/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766837 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766812.683048:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4190
00000100:00080000:3.0:1775766812.683052:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766812.683058:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766812.683065:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766812.683092:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745c00 x1862022114578816/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775766812.683098:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746680 x1862022114578176/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766837 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766812.683102:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766812.683107:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775766812.683109:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766812.683117:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4190
00000100:00080000:3.0:1775766812.683124:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766812.683127:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00080000:0.0:1775766812.683128:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000008:00100000:3.0:1775766812.683130:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00080000:0.0:1775766812.683137:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766812.683141:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745180 x1862022114578944/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775766812.683145:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114578176:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775766812.683157:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766812.683159:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775766812.683160:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766812.683164:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4190
00000100:00080000:3.0:1775766812.683166:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766812.683168:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766812.683172:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775766812.683176:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114578304:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766812.683181:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745f80 x1862022114579072/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1775766812.683228:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766812.683240:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766812/real 1775766812]  req@ffff9f1485744700 x1862022114578304/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766837 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766812.683249:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744700 x1862022114578304/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766837 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766812.683255:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766812.683257:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766812.683259:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114578304:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766812.683263:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114578432:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766812.683285:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766812.683292:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766812/real 1775766812]  req@ffff9f1485745500 x1862022114578432/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766837 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766812.683300:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745500 x1862022114578432/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766837 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766812.683305:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766812.683307:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766812.683309:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114578432:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766812.683313:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114578560:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766812.683324:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766812.683329:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766812/real 1775766812]  req@ffff9f1485744000 x1862022114578560/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766837 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766812.683336:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744000 x1862022114578560/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766837 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766812.683341:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766812.683343:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766812.683344:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114578560:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775766812.683348:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114578688:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766812.683357:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766812.683362:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766812/real 1775766812]  req@ffff9f1485747800 x1862022114578688/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766837 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766812.683374:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747800 x1862022114578688/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766837 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766812.683379:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766812.683381:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766812.683382:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114578688:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775766812.683391:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114578816:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766812.683402:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766812.683408:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766812/real 1775766812]  req@ffff9f1485745c00 x1862022114578816/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766837 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766812.683415:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745c00 x1862022114578816/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766837 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766812.683420:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766812.683422:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766812.683423:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114578816:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766812.683427:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114578944:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766812.683436:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766812.683447:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766812/real 1775766812]  req@ffff9f1485745180 x1862022114578944/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766837 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766812.683454:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745180 x1862022114578944/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766837 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766812.683464:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766812.683466:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766812.683468:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114578944:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766812.683480:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114579072:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766812.683490:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766812.683496:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766812/real 1775766812]  req@ffff9f1485745f80 x1862022114579072/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766837 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766812.683503:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745f80 x1862022114579072/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766837 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766812.683508:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766812.683510:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766812.683512:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114579072:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775766812.938738:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766813.706858:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766813.962805:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766814.730745:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775766814.986750:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766815.754667:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766816.010749:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766816.778735:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766817.034742:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766817.802713:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:1.0:1775766817.802827:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766817.802841:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1775766817.802850:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766817.802867:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4195
00000100:00080000:1.0:1775766817.802887:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766817.802900:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775766817.802970:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdf80 x1862022114579200/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766817.803027:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766817.803035:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775766817.803039:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766817.803047:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4195
00000100:00080000:1.0:1775766817.803074:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766817.803084:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766817.803105:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775766817.803135:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114579200:192.168.202.110@tcp:250:kworker.0
00000100:00100000:1.0:1775766817.803136:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff800 x1862022114579328/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766817.803183:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766817.803188:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775766817.803204:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766817.803211:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4195
00000100:00080000:1.0:1775766817.803218:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766817.803224:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766817.803232:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775766817.803248:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fea00 x1862022114579456/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766817.803261:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766817.803265:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775766817.803267:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766817.803276:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4195
00000800:00000100:0.0:1775766817.803277:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:1.0:1775766817.803281:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766817.803287:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766817.803293:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775766817.803308:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ffb80 x1862022114579584/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766817.803329:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766817.803333:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1775766817.803336:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766817.803341:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4195
00000100:00080000:1.0:1775766817.803346:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1775766817.803349:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766817/real 1775766817]  req@ffff9f14ab4fdf80 x1862022114579200/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766842 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766817.803355:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766817.803367:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775766817.803390:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fce00 x1862022114579712/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775766817.803397:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdf80 x1862022114579200/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766842 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766817.803420:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775766817.803423:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766817.803431:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775766817.803432:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:1.0:1775766817.803435:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775766817.803442:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114579200:192.168.202.110@tcp:250:kworker.0
00000100:00080000:1.0:1775766817.803447:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4195
00000100:00080000:1.0:1775766817.803457:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766817.803464:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766817.803471:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775766817.803484:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fca80 x1862022114579840/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775766817.803492:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114579328:192.168.202.110@tcp:38:kworker.0
00000100:00080000:1.0:1775766817.803515:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766817.803520:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775766817.803523:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766817.803530:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4195
00000100:00080000:1.0:1775766817.803537:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000800:00000100:0.0:1775766817.803542:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:1.0:1775766817.803547:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766817.803600:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775766817.803619:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fed80 x1862022114579968/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766817.803630:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766817.803635:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00100000:0.0:1775766817.803638:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766817/real 1775766817]  req@ffff9f14ab4ff800 x1862022114579328/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766842 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766817.803638:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766817.803651:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4195
00000100:00080000:1.0:1775766817.803659:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766817.803670:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766817.803683:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775766817.803698:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff800 x1862022114579328/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766842 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775766817.803701:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdc00 x1862022114580096/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775766817.803752:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766817.803762:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766817.803768:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114579328:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766817.803810:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114579456:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766817.803842:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766817.803857:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766817/real 1775766817]  req@ffff9f14ab4fea00 x1862022114579456/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766842 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766817.803872:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fea00 x1862022114579456/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766842 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766817.803882:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766817.803886:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766817.803890:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114579456:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766817.803896:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ffb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114579584:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766817.803916:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766817.803926:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766817/real 1775766817]  req@ffff9f14ab4ffb80 x1862022114579584/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766842 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766817.803942:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ffb80 x1862022114579584/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766842 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766817.803957:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766817.803963:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766817.803968:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ffb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114579584:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775766817.803979:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114579712:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766817.804014:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766817.804051:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766817/real 1775766817]  req@ffff9f14ab4fce00 x1862022114579712/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766842 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766817.804076:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fce00 x1862022114579712/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766842 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766817.804089:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766817.804093:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766817.804097:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114579712:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775766817.804131:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114579840:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766817.804173:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766817.804196:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766817/real 1775766817]  req@ffff9f14ab4fca80 x1862022114579840/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766842 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766817.804237:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fca80 x1862022114579840/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766842 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766817.804259:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766817.804263:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766817.804267:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114579840:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766817.804274:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114579968:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766817.804300:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766817.804313:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766817/real 1775766817]  req@ffff9f14ab4fed80 x1862022114579968/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766842 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766817.804327:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fed80 x1862022114579968/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766842 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766817.804336:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766817.804340:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766817.804343:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114579968:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766817.804349:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114580096:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766817.804366:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766817.804375:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766817/real 1775766817]  req@ffff9f14ab4fdc00 x1862022114580096/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766842 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766817.804388:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdc00 x1862022114580096/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766842 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766817.804397:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766817.804400:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766817.804404:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114580096:192.168.202.110@tcp:8:kworker.0
00010000:00010000:3.0:1775766818.058722:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766818.826673:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:0.0:1775766819.082744:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766819.850689:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766820.106705:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766820.874663:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766821.130782:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766821.898667:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775766822.154782:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766822.922670:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:1.0:1775766822.922696:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766822.922704:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1775766822.922709:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766822.922729:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4200
00000100:00080000:1.0:1775766822.922742:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766822.922750:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775766822.922795:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff100 x1862022114580224/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766822.922820:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766822.922823:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775766822.922825:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766822.922828:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4200
00000100:00080000:1.0:1775766822.922832:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766822.922834:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766822.922845:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775766822.922858:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114580224:192.168.202.110@tcp:250:kworker.0
00000100:00100000:1.0:1775766822.922859:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fe300 x1862022114580352/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766822.922885:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766822.922889:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775766822.922891:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766822.922897:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4200
00000100:00080000:1.0:1775766822.922902:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766822.922906:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766822.922912:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775766822.922928:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fd880 x1862022114580480/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766822.922938:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766822.922942:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775766822.922943:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766822.922947:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4200
00000100:00080000:1.0:1775766822.922951:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766822.922953:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000800:00000100:0.0:1775766822.922963:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000008:00100000:1.0:1775766822.922966:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775766822.922980:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fc380 x1862022114580608/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766822.922988:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766822.922990:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1775766822.922992:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766822.922995:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4200
00000100:00080000:1.0:1775766822.923000:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766822.923004:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766822.923008:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775766822.923013:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766822/real 1775766822]  req@ffff9f14ab4ff100 x1862022114580224/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766847 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775766822.923020:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdc00 x1862022114580736/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766822.923059:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766822.923062:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775766822.923064:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775766822.923071:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff100 x1862022114580224/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766847 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766822.923071:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4200
00000100:00080000:1.0:1775766822.923082:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766822.923094:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766822.923098:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00080000:0.0:1775766822.923099:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766822.923105:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775766822.923111:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fed80 x1862022114580864/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775766822.923112:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114580224:192.168.202.110@tcp:250:kworker.0
00000100:00080000:1.0:1775766822.923150:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766822.923153:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775766822.923155:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775766822.923166:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114580352:192.168.202.110@tcp:38:kworker.0
00000100:00080000:1.0:1775766822.923167:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4200
00000100:00080000:1.0:1775766822.923180:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766822.923186:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766822.923191:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000800:00000100:0.0:1775766822.923200:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775766822.923204:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fca80 x1862022114580992/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775766822.923224:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766822/real 1775766822]  req@ffff9f14ab4fe300 x1862022114580352/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766847 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766822.923263:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766822.923266:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00100000:0.0:1775766822.923269:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fe300 x1862022114580352/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766847 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766822.923270:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766822.923286:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775766822.923292:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4200
00000100:00080000:0.0:1775766822.923299:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00080000:1.0:1775766822.923301:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1775766822.923305:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114580352:192.168.202.110@tcp:38:kworker.0
00000100:00080000:1.0:1775766822.923326:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775766822.923328:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114580480:192.168.202.110@tcp:38:kworker.0
00000008:00100000:1.0:1775766822.923336:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775766822.923352:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fce00 x1862022114581120/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1775766822.923364:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766822.923394:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766822/real 1775766822]  req@ffff9f14ab4fd880 x1862022114580480/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766847 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766822.923411:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fd880 x1862022114580480/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766847 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766822.923420:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766822.923425:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766822.923427:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114580480:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766822.923433:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114580608:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766822.923451:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766822.923459:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766822/real 1775766822]  req@ffff9f14ab4fc380 x1862022114580608/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766847 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766822.923467:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fc380 x1862022114580608/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766847 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766822.923472:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766822.923475:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766822.923476:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114580608:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775766822.923480:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114580736:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766822.923490:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766822.923495:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766822/real 1775766822]  req@ffff9f14ab4fdc00 x1862022114580736/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766847 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766822.923507:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdc00 x1862022114580736/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766847 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766822.923512:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766822.923514:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766822.923516:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114580736:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775766822.923519:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114580864:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766822.923529:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766822.923534:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766822/real 1775766822]  req@ffff9f14ab4fed80 x1862022114580864/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766847 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766822.923541:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fed80 x1862022114580864/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766847 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766822.923546:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766822.923548:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766822.923550:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114580864:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766822.923612:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114580992:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766822.923625:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766822.923632:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766822/real 1775766822]  req@ffff9f14ab4fca80 x1862022114580992/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766847 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766822.923640:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fca80 x1862022114580992/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766847 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766822.923650:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766822.923652:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766822.923654:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114580992:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766822.923658:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114581120:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766822.923668:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766822.923673:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766822/real 1775766822]  req@ffff9f14ab4fce00 x1862022114581120/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766847 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766822.923680:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fce00 x1862022114581120/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766847 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766822.923685:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766822.923687:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766822.923688:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114581120:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775766823.178800:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766823.946610:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766824.202753:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766824.970644:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775766825.226928:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766825.994726:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:0.0:1775766826.250852:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766827.018706:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766827.274792:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766828.042640:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775766828.042697:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766828.042704:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775766828.042709:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766828.042719:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4205
00000100:00080000:3.0:1775766828.042731:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766828.042739:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775766828.042780:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747100 x1862022114581248/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766828.042805:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766828.042808:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775766828.042809:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766828.042812:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4205
00000100:00080000:3.0:1775766828.042815:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766828.042818:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766828.042826:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766828.042837:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745f80 x1862022114581376/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766828.042844:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766828.042847:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775766828.042848:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766828.042850:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4205
00000100:00080000:3.0:1775766828.042852:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766828.042871:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766828.042875:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775766828.042878:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114581248:192.168.202.110@tcp:250:kworker.0
00000100:00100000:3.0:1775766828.042884:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745180 x1862022114581504/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766828.042897:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766828.042899:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775766828.042901:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766828.042904:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4205
00000100:00080000:3.0:1775766828.042906:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766828.042908:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766828.042911:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766828.042919:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745c00 x1862022114581632/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766828.042925:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766828.042927:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775766828.042929:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766828.042931:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4205
00000100:00080000:3.0:1775766828.042934:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766828.042936:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766828.042938:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766828.042946:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747800 x1862022114581760/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766828.042951:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766828.042955:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000800:00000100:0.0:1775766828.042956:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775766828.042957:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766828.042963:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4205
00000100:00080000:3.0:1775766828.042966:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766828.042969:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766828.042972:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766828.042980:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744000 x1862022114581888/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766828.042986:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766828.042988:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775766828.042989:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766828.042992:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4205
00000100:00080000:3.0:1775766828.042995:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766828.042998:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766828.043002:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775766828.043006:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766828/real 1775766828]  req@ffff9f1485747100 x1862022114581248/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766853 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766828.043027:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745500 x1862022114582016/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766828.043046:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766828.043049:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775766828.043050:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766828.043053:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4205
00000100:00080000:3.0:1775766828.043056:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1775766828.043059:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747100 x1862022114581248/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766853 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766828.043060:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766828.043066:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00080000:0.0:1775766828.043077:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00100000:3.0:1775766828.043079:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744700 x1862022114582144/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775766828.043083:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766828.043092:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114581248:192.168.202.110@tcp:250:kworker.0
00000100:00100000:0.0:1775766828.043105:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114581376:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766828.043133:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766828.043147:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766828/real 1775766828]  req@ffff9f1485745f80 x1862022114581376/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766853 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766828.043159:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745f80 x1862022114581376/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766853 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766828.043165:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766828.043167:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766828.043169:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114581376:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766828.043188:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114581504:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766828.043202:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766828.043209:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766828/real 1775766828]  req@ffff9f1485745180 x1862022114581504/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766853 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766828.043217:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745180 x1862022114581504/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766853 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766828.043222:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766828.043224:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766828.043226:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114581504:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766828.043230:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114581632:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766828.043240:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766828.043246:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766828/real 1775766828]  req@ffff9f1485745c00 x1862022114581632/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766853 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766828.043253:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745c00 x1862022114581632/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766853 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766828.043258:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766828.043260:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766828.043261:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114581632:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775766828.043264:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114581760:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766828.043273:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766828.043278:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766828/real 1775766828]  req@ffff9f1485747800 x1862022114581760/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766853 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766828.043286:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747800 x1862022114581760/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766853 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766828.043291:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766828.043292:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766828.043294:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114581760:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775766828.043297:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114581888:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766828.043306:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766828.043311:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766828/real 1775766828]  req@ffff9f1485744000 x1862022114581888/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766853 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766828.043318:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744000 x1862022114581888/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766853 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766828.043324:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766828.043326:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766828.043328:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114581888:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766828.043331:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114582016:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766828.043340:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766828.043345:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766828/real 1775766828]  req@ffff9f1485745500 x1862022114582016/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766853 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766828.043351:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745500 x1862022114582016/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766853 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766828.043356:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766828.043358:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766828.043360:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114582016:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766828.043363:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114582144:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766828.043372:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766828.043377:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766828/real 1775766828]  req@ffff9f1485744700 x1862022114582144/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766853 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766828.043383:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744700 x1862022114582144/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766853 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766828.043388:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766828.043389:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766828.043392:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114582144:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775766828.298705:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766829.066681:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775766829.322717:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766830.090722:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766830.346700:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766831.114643:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766831.370738:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766832.138746:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775766832.394809:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1775766833.162673:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:2.0:1775766833.162673:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:0.0:1775766833.162685:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1775766833.162690:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766833.162710:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4210
00000100:00080000:0.0:1775766833.162722:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766833.162730:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775766833.162780:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3800 x1862022114582272/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775766833.162812:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766833.162816:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775766833.162818:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766833.162823:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4210
00000100:00080000:0.0:1775766833.162828:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766833.162834:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775766833.162845:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766833.162851:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114582272:192.168.202.110@tcp:250:kworker.0
00000100:00100000:0.0:1775766833.162864:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d1500 x1862022114582400/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775766833.162878:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766833.162882:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:0.0:1775766833.162885:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766833.162890:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4210
00000100:00080000:0.0:1775766833.162894:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766833.162899:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775766833.162905:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000800:00000100:2.0:1775766833.162920:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766833.162921:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0000 x1862022114582528/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775766833.162935:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766833.162940:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775766833.162942:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766833.162946:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4210
00000100:00080000:0.0:1775766833.162951:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766833.162955:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775766833.162962:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775766833.162966:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766833/real 1775766833]  req@ffff9f14a04d3800 x1862022114582272/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766858 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766833.162978:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d2300 x1862022114582656/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775766833.162985:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3800 x1862022114582272/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766858 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766833.162999:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766833.163001:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766833.163003:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1775766833.163007:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766833.163007:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766833.163014:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114582272:192.168.202.110@tcp:250:kworker.0
00000100:00080000:0.0:1775766833.163016:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4210
00000100:00080000:0.0:1775766833.163024:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766833.163037:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775766833.163040:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114582400:192.168.202.110@tcp:38:kworker.0
00000008:00100000:0.0:1775766833.163045:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775766833.163061:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3b80 x1862022114582784/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:2.0:1775766833.163072:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:0.0:1775766833.163075:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766833.163080:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775766833.163083:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:2.0:1775766833.163086:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766833/real 1775766833]  req@ffff9f14a04d1500 x1862022114582400/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766858 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766833.163092:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4210
00000100:00080000:0.0:1775766833.163100:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:2.0:1775766833.163102:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1500 x1862022114582400/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766858 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766833.163108:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00080000:2.0:1775766833.163116:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000008:00100000:0.0:1775766833.163118:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00080000:2.0:1775766833.163122:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766833.163125:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114582400:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766833.163134:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3100 x1862022114582912/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775766833.163135:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114582528:192.168.202.110@tcp:38:kworker.0
00000100:00080000:0.0:1775766833.163149:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766833.163152:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:0.0:1775766833.163154:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000800:00000100:2.0:1775766833.163159:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:0.0:1775766833.163163:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4210
00000100:00080000:0.0:1775766833.163167:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766833.163170:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775766833.163174:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766833/real 1775766833]  req@ffff9f14a04d0000 x1862022114582528/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766858 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:0.0:1775766833.163175:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775766833.163187:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0a80 x1862022114583040/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775766833.163190:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0000 x1862022114582528/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766858 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766833.163205:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766833.163209:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766833.163210:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775766833.163213:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766833.163217:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00080000:0.0:1775766833.163220:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4210
00000100:00080000:0.0:1775766833.163224:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:2.0:1775766833.163224:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114582528:192.168.202.110@tcp:38:kworker.0
00000100:00080000:0.0:1775766833.163231:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775766833.163235:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114582656:192.168.202.110@tcp:8:kworker.0
00000008:00100000:0.0:1775766833.163237:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775766833.163249:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d2a00 x1862022114583168/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:2.0:1775766833.163257:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766833.163266:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766833/real 1775766833]  req@ffff9f14a04d2300 x1862022114582656/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766858 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766833.163276:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d2300 x1862022114582656/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766858 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766833.163282:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766833.163285:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766833.163287:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114582656:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775766833.163300:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114582784:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775766833.163311:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766833.163316:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766833/real 1775766833]  req@ffff9f14a04d3b80 x1862022114582784/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766858 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766833.163324:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3b80 x1862022114582784/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766858 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766833.163329:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766833.163331:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766833.163333:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114582784:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775766833.163337:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114582912:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775766833.163347:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766833.163352:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766833/real 1775766833]  req@ffff9f14a04d3100 x1862022114582912/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766858 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766833.163365:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3100 x1862022114582912/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766858 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766833.163370:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766833.163372:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766833.163374:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114582912:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766833.163377:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114583040:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775766833.163387:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766833.163392:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766833/real 1775766833]  req@ffff9f14a04d0a80 x1862022114583040/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766858 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766833.163400:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0a80 x1862022114583040/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766858 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766833.163404:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766833.163406:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766833.163408:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114583040:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766833.163411:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114583168:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775766833.163421:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766833.163426:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766833/real 1775766833]  req@ffff9f14a04d2a00 x1862022114583168/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766858 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766833.163432:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d2a00 x1862022114583168/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766858 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766833.163442:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766833.163444:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766833.163446:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114583168:192.168.202.110@tcp:8:kworker.0
00010000:00010000:0.0:1775766833.418730:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766834.186700:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766834.442724:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766835.210660:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766835.466717:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766836.234692:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775766836.490799:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766837.258662:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766837.514801:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766838.282707:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:2.0:1775766838.282775:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766838.282782:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1775766838.282788:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766838.282820:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4215
00000100:00080000:2.0:1775766838.282835:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766838.282845:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775766838.282911:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2d80 x1862022114583296/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766838.282951:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766838.282957:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775766838.282960:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766838.282967:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4215
00000100:00080000:2.0:1775766838.282973:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766838.282979:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775766838.282985:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114583296:192.168.202.110@tcp:250:kworker.0
00000008:00100000:2.0:1775766838.282996:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766838.283028:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0380 x1862022114583424/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766838.283053:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766838.283057:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775766838.283059:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766838.283064:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4215
00000100:00080000:2.0:1775766838.283070:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766838.283074:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000800:00000100:3.0:1775766838.283080:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000008:00100000:2.0:1775766838.283081:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766838.283097:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1500 x1862022114583552/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766838.283108:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766838.283112:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775766838.283114:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766838.283130:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4215
00000100:00080000:2.0:1775766838.283145:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766838.283150:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766838.283157:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775766838.283171:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1f80 x1862022114583680/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766838.283182:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766838.283186:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1775766838.283188:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766838.283193:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4215
00000100:00080000:2.0:1775766838.283199:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00100000:3.0:1775766838.283199:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766838/real 1775766838]  req@ffff9f14852d2d80 x1862022114583296/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766863 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766838.283209:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766838.283219:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766838.283226:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2d80 x1862022114583296/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766863 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766838.283235:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0a80 x1862022114583808/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766838.283247:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766838.283248:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766838.283255:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775766838.283257:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:2.0:1775766838.283258:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:3.0:1775766838.283266:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114583296:192.168.202.110@tcp:250:kworker.0
00000100:00080000:2.0:1775766838.283267:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4215
00000100:00080000:2.0:1775766838.283276:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766838.283283:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766838.283289:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766838.283302:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2680 x1862022114583936/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775766838.283305:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114583424:192.168.202.110@tcp:38:kworker.0
00000100:00080000:2.0:1775766838.283331:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766838.283335:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775766838.283337:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766838.283343:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4215
00000100:00080000:2.0:1775766838.283348:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766838.283355:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766838.283361:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766838.283375:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1180 x1862022114584064/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766838.283395:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766838.283398:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775766838.283400:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766838.283405:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4215
00000100:00080000:2.0:1775766838.283409:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766838.283413:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766838.283418:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775766838.283429:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2a00 x1862022114584192/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:3.0:1775766838.283499:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766838.283514:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766838/real 1775766838]  req@ffff9f14852d0380 x1862022114583424/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766863 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766838.283634:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0380 x1862022114583424/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766863 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766838.283644:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766838.283653:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766838.283657:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114583424:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766838.283665:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114583552:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775766838.283696:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766838.283705:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766838/real 1775766838]  req@ffff9f14852d1500 x1862022114583552/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766863 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766838.283713:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1500 x1862022114583552/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766863 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766838.283719:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766838.283721:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766838.283723:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114583552:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766838.283727:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114583680:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775766838.283738:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766838.283744:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766838/real 1775766838]  req@ffff9f14852d1f80 x1862022114583680/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766863 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766838.283751:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1f80 x1862022114583680/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766863 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766838.283757:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766838.283758:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766838.283760:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114583680:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775766838.283763:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114583808:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775766838.283774:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766838.283779:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766838/real 1775766838]  req@ffff9f14852d0a80 x1862022114583808/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766863 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766838.283792:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0a80 x1862022114583808/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766863 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766838.283797:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766838.283798:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766838.283800:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114583808:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775766838.283824:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114583936:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775766838.283835:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766838.283841:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766838/real 1775766838]  req@ffff9f14852d2680 x1862022114583936/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766863 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766838.283849:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2680 x1862022114583936/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766863 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766838.283854:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766838.283856:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766838.283857:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114583936:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766838.283861:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114584064:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775766838.283887:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766838.283894:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766838/real 1775766838]  req@ffff9f14852d1180 x1862022114584064/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766863 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766838.283901:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1180 x1862022114584064/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766863 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766838.283912:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766838.283914:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766838.283916:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114584064:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766838.283919:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114584192:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775766838.283929:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766838.283934:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766838/real 1775766838]  req@ffff9f14852d2a00 x1862022114584192/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766863 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766838.283941:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2a00 x1862022114584192/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766863 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766838.283947:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766838.283948:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766838.283950:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114584192:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775766838.538822:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766839.306630:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775766839.562742:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766840.330749:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:0.0:1775766840.586745:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766841.354679:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766841.610800:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766842.378688:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766842.634784:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1775766843.402639:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766843.402646:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775766843.402650:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766843.402659:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4220
00000100:00100000:2.0:1775766843.402665:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775766843.402670:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766843.402680:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775766843.402712:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746680 x1862022114584320/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766843.402739:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766843.402743:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775766843.402744:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766843.402748:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4220
00000100:00080000:3.0:1775766843.402751:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766843.402754:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766843.402761:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766843.402772:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744a80 x1862022114584448/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775766843.402782:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114584320:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775766843.402783:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766843.402794:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775766843.402796:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766843.402801:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4220
00000100:00080000:3.0:1775766843.402803:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766843.402805:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766843.402810:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766843.402823:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747480 x1862022114584576/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766843.402832:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766843.402836:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775766843.402839:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766843.402843:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4220
00000100:00080000:3.0:1775766843.402856:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766843.402863:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000800:00000100:2.0:1775766843.402869:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000008:00100000:3.0:1775766843.402875:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766843.402897:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745880 x1862022114584704/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766843.402909:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766843.402911:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775766843.402914:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766843.402919:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4220
00000100:00100000:2.0:1775766843.402925:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766843/real 1775766843]  req@ffff9f1485746680 x1862022114584320/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766868 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766843.402928:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766843.402947:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766843.402955:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775766843.402956:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746680 x1862022114584320/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766868 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766843.402978:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746300 x1862022114584832/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766843.402996:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766843.402999:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766843.403003:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775766843.403006:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766843.403007:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:3.0:1775766843.403012:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4220
00000100:00080000:3.0:1775766843.403016:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:2.0:1775766843.403020:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114584320:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775766843.403026:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766843.403038:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766843.403052:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746a00 x1862022114584960/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766843.403065:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:2.0:1775766843.403066:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114584448:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775766843.403080:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775766843.403083:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766843.403090:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4220
00000100:00080000:3.0:1775766843.403095:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766843.403100:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766843.403107:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000800:00000100:2.0:1775766843.403115:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766843.403126:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744380 x1862022114585088/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766843.403135:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:2.0:1775766843.403142:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766843/real 1775766843]  req@ffff9f1485744a80 x1862022114584448/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766868 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766843.403143:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775766843.403147:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766843.403160:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4220
00000100:00080000:3.0:1775766843.403171:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766843.403176:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766843.403181:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775766843.403182:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744a80 x1862022114584448/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766868 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766843.403200:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00100000:3.0:1775766843.403201:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744e00 x1862022114585216/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766843.403210:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766843.403227:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114584448:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766843.403244:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114584576:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775766843.403269:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766843.403278:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766843/real 1775766843]  req@ffff9f1485747480 x1862022114584576/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766868 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766843.403287:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747480 x1862022114584576/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766868 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766843.403293:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766843.403296:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766843.403298:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114584576:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766843.403304:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114584704:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775766843.403317:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766843.403324:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766843/real 1775766843]  req@ffff9f1485745880 x1862022114584704/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766868 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766843.403333:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745880 x1862022114584704/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766868 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766843.403339:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766843.403341:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766843.403343:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114584704:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775766843.403348:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114584832:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775766843.403360:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766843.403366:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766843/real 1775766843]  req@ffff9f1485746300 x1862022114584832/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766868 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766843.403375:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746300 x1862022114584832/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766868 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766843.403380:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766843.403383:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766843.403385:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114584832:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775766843.403403:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114584960:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775766843.403427:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766843.403440:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766843/real 1775766843]  req@ffff9f1485746a00 x1862022114584960/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766868 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766843.403456:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746a00 x1862022114584960/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766868 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766843.403467:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766843.403471:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766843.403474:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114584960:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766843.403480:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114585088:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775766843.403525:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766843.403536:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766843/real 1775766843]  req@ffff9f1485744380 x1862022114585088/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766868 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766843.403546:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744380 x1862022114585088/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766868 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766843.403583:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766843.403587:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766843.403590:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114585088:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766843.403600:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114585216:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775766843.403620:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766843.403640:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766843/real 1775766843]  req@ffff9f1485744e00 x1862022114585216/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766868 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766843.403651:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744e00 x1862022114585216/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766868 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766843.403665:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766843.403669:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766843.403673:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114585216:192.168.202.110@tcp:8:kworker.0
00010000:00010000:1.0:1775766843.658773:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766844.426680:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766844.682760:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766845.450681:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766845.706730:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766846.474655:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775766846.730738:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766847.498731:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:0.0:1775766847.754709:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766848.522681:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:2.0:1775766848.522750:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766848.522759:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1775766848.522764:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766848.522776:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4226
00000100:00080000:2.0:1775766848.522792:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766848.522816:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775766848.522868:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3800 x1862022114585344/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766848.522917:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766848.522921:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775766848.522922:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766848.522926:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4226
00000100:00080000:2.0:1775766848.522929:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766848.522932:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766848.522943:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766848.522956:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0700 x1862022114585472/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766848.522979:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766848.522983:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775766848.522984:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766848.522990:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4226
00000100:00080000:2.0:1775766848.522994:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766848.522997:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766848.523001:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766848.523015:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2a00 x1862022114585600/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775766848.523016:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114585344:192.168.202.110@tcp:250:kworker.0
00000100:00080000:2.0:1775766848.523059:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766848.523062:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775766848.523064:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766848.523068:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4226
00000100:00080000:2.0:1775766848.523070:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766848.523073:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766848.523079:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775766848.523090:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1180 x1862022114585728/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766848.523096:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766848.523099:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1775766848.523101:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766848.523105:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4226
00000100:00080000:2.0:1775766848.523108:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766848.523112:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766848.523115:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775766848.523125:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2680 x1862022114585856/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766848.523130:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766848.523133:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775766848.523134:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766848.523138:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4226
00000100:00080000:2.0:1775766848.523141:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766848.523145:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766848.523149:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766848.523169:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0a80 x1862022114585984/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766848.523174:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766848.523180:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775766848.523186:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000800:00000100:3.0:1775766848.523193:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:2.0:1775766848.523194:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4226
00000100:00080000:2.0:1775766848.523202:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766848.523208:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766848.523212:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766848.523222:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1f80 x1862022114586112/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766848.523234:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766848.523238:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775766848.523240:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766848.523243:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4226
00000100:00080000:2.0:1775766848.523246:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766848.523248:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766848.523255:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775766848.523265:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1500 x1862022114586240/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775766848.523296:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766848/real 1775766848]  req@ffff9f14852d3800 x1862022114585344/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766873 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766848.523324:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3800 x1862022114585344/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766873 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766848.523347:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766848.523354:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766848.523362:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114585344:192.168.202.110@tcp:250:kworker.0
00000100:00100000:3.0:1775766848.523377:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114585472:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775766848.523503:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766848.523528:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766848/real 1775766848]  req@ffff9f14852d0700 x1862022114585472/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766873 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766848.523552:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0700 x1862022114585472/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766873 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766848.523563:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766848.523568:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766848.523603:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114585472:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766848.523636:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114585600:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775766848.523665:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766848.523679:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766848/real 1775766848]  req@ffff9f14852d2a00 x1862022114585600/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766873 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766848.523693:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2a00 x1862022114585600/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766873 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766848.523702:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766848.523706:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766848.523710:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114585600:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766848.523717:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114585728:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775766848.523736:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766848.523746:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766848/real 1775766848]  req@ffff9f14852d1180 x1862022114585728/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766873 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766848.523763:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1180 x1862022114585728/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766873 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766848.523778:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766848.523784:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766848.523789:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114585728:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775766848.523800:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114585856:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775766848.523834:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766848.523853:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766848/real 1775766848]  req@ffff9f14852d2680 x1862022114585856/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766873 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766848.523892:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2680 x1862022114585856/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766873 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766848.523902:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766848.523906:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766848.523910:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114585856:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775766848.523918:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114585984:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775766848.523946:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766848.523977:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766848/real 1775766848]  req@ffff9f14852d0a80 x1862022114585984/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766873 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766848.523992:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0a80 x1862022114585984/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766873 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766848.524001:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766848.524005:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766848.524008:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114585984:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766848.524014:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114586112:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775766848.524033:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766848.524043:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766848/real 1775766848]  req@ffff9f14852d1f80 x1862022114586112/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766873 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766848.524056:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1f80 x1862022114586112/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766873 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766848.524065:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766848.524070:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766848.524075:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114586112:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766848.524085:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114586240:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775766848.524119:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766848.524139:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766848/real 1775766848]  req@ffff9f14852d1500 x1862022114586240/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766873 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766848.524161:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1500 x1862022114586240/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766873 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766848.524190:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766848.524196:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766848.524202:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114586240:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775766848.778784:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766849.546736:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766849.802745:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766850.570637:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775766850.826715:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766851.594688:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766851.850697:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766852.618676:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766852.874785:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766853.642688:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775766853.642704:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766853.642714:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775766853.642719:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766853.642733:0:8:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4231
00000100:00080000:3.0:1775766853.642748:0:8:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766853.642759:0:8:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775766853.642805:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747b80 x1862022114586368/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766853.642835:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766853.642838:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775766853.642841:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766853.642847:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4231
00000100:00080000:3.0:1775766853.642854:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766853.642861:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775766853.642873:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114586368:192.168.202.110@tcp:250:kworker.0
00000008:00100000:3.0:1775766853.642874:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766853.642904:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744e00 x1862022114586496/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766853.642922:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766853.642927:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775766853.642930:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766853.642934:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4231
00000100:00080000:3.0:1775766853.642941:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766853.642945:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766853.642951:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766853.642966:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744380 x1862022114586624/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:2.0:1775766853.642970:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775766853.642987:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766853.642992:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775766853.642995:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766853.643002:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4231
00000100:00080000:3.0:1775766853.643010:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766853.643017:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775766853.643023:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766853/real 1775766853]  req@ffff9f1485747b80 x1862022114586368/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766878 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:3.0:1775766853.643025:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766853.643051:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746a00 x1862022114586752/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775766853.643055:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747b80 x1862022114586368/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766878 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766853.643080:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766853.643086:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766853.643094:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114586368:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775766853.643094:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766853.643107:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775766853.643109:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766853.643115:0:8:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4231
00000100:00080000:3.0:1775766853.643120:0:8:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766853.643124:0:8:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766853.643129:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775766853.643137:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114586496:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766853.643144:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746300 x1862022114586880/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766853.643158:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766853.643162:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775766853.643164:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766853.643167:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4231
00000100:00080000:3.0:1775766853.643179:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000800:00000100:2.0:1775766853.643187:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775766853.643191:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766853.643197:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766853.643209:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745880 x1862022114587008/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775766853.643217:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766853/real 1775766853]  req@ffff9f1485744e00 x1862022114586496/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766878 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766853.643224:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766853.643235:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775766853.643239:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766853.643248:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4231
00000100:00080000:3.0:1775766853.643253:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:2.0:1775766853.643257:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744e00 x1862022114586496/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766878 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766853.643260:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766853.643270:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00080000:2.0:1775766853.643284:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00100000:3.0:1775766853.643286:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747480 x1862022114587136/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766853.643300:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00080000:3.0:1775766853.643302:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766853.643308:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775766853.643311:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:2.0:1775766853.643313:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114586496:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775766853.643319:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4231
00000100:00080000:3.0:1775766853.643327:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766853.643331:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775766853.643336:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114586624:192.168.202.110@tcp:38:kworker.0
00000008:00100000:3.0:1775766853.643337:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766853.643350:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744a80 x1862022114587264/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:2.0:1775766853.643387:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766853.643422:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766853/real 1775766853]  req@ffff9f1485744380 x1862022114586624/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766878 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766853.643434:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744380 x1862022114586624/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766878 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766853.643442:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766853.643445:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766853.643448:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114586624:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766853.643454:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114586752:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775766853.643474:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766853.643483:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766853/real 1775766853]  req@ffff9f1485746a00 x1862022114586752/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766878 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766853.643500:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746a00 x1862022114586752/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766878 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766853.643507:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766853.643509:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766853.643512:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114586752:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775766853.643528:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114586880:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775766853.643542:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766853.643550:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766853/real 1775766853]  req@ffff9f1485746300 x1862022114586880/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766878 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766853.643595:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746300 x1862022114586880/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766878 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766853.643606:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766853.643611:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766853.643616:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114586880:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775766853.643625:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114587008:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775766853.643653:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766853.643668:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766853/real 1775766853]  req@ffff9f1485745880 x1862022114587008/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766878 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766853.643679:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745880 x1862022114587008/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766878 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766853.643693:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766853.643695:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766853.643699:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114587008:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766853.643703:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114587136:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775766853.643720:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766853.643731:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766853/real 1775766853]  req@ffff9f1485747480 x1862022114587136/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766878 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766853.643746:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747480 x1862022114587136/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766878 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766853.643757:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766853.643762:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766853.643766:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114587136:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766853.643776:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114587264:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775766853.643805:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766853.643817:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766853/real 1775766853]  req@ffff9f1485744a80 x1862022114587264/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766878 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766853.643828:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744a80 x1862022114587264/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766878 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766853.643836:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766853.643838:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766853.643842:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114587264:192.168.202.110@tcp:8:kworker.0
00010000:00010000:3.0:1775766853.898780:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766854.666640:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:0.0:1775766854.922728:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766855.690671:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766855.946805:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766856.714727:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766856.970723:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766857.738655:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775766857.994741:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766858.762628:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775766858.762628:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766858.762637:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775766858.762641:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766858.762658:0:8:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4236
00000100:00080000:3.0:1775766858.762668:0:8:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766858.762674:0:8:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775766858.762707:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746680 x1862022114587392/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766858.762729:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766858.762732:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775766858.762733:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766858.762736:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4236
00000100:00080000:3.0:1775766858.762738:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766858.762740:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766858.762748:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766858.762760:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744700 x1862022114587520/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766858.762772:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766858.762775:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775766858.762776:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766858.762779:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4236
00000100:00080000:3.0:1775766858.762781:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766858.762784:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766858.762786:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766858.762793:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745500 x1862022114587648/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766858.762798:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766858.762801:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775766858.762802:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:2.0:1775766858.762807:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114587392:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775766858.762807:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4236
00000100:00080000:3.0:1775766858.762812:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766858.762817:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766858.762820:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766858.762827:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744000 x1862022114587776/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766858.762832:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766858.762834:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775766858.762836:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766858.762838:0:8:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4236
00000100:00080000:3.0:1775766858.762842:0:8:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766858.762844:0:8:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766858.762848:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766858.762854:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747800 x1862022114587904/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766858.762860:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766858.762862:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775766858.762864:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766858.762867:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4236
00000100:00080000:3.0:1775766858.762869:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766858.762872:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766858.762875:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000800:00000100:2.0:1775766858.762878:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766858.762886:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745c00 x1862022114588032/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766858.762892:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766858.762895:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775766858.762897:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766858.762899:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4236
00000100:00080000:3.0:1775766858.762902:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766858.762909:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766858.762912:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766858.762919:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766858/real 1775766858]  req@ffff9f1485746680 x1862022114587392/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766883 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766858.762922:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745180 x1862022114588160/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766858.762953:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766858.762955:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00100000:2.0:1775766858.762957:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746680 x1862022114587392/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766883 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766858.762958:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766858.762965:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4236
00000100:00080000:3.0:1775766858.762971:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766858.762976:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00080000:2.0:1775766858.762978:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000008:00100000:3.0:1775766858.762979:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00080000:2.0:1775766858.762987:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766858.762991:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745f80 x1862022114588288/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775766858.762994:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114587392:192.168.202.110@tcp:250:kworker.0
00000100:00100000:2.0:1775766858.763007:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114587520:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775766858.763025:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766858.763033:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766858/real 1775766858]  req@ffff9f1485744700 x1862022114587520/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766883 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766858.763042:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744700 x1862022114587520/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766883 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766858.763047:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766858.763050:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766858.763052:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114587520:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766858.763070:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114587648:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775766858.763081:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766858.763087:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766858/real 1775766858]  req@ffff9f1485745500 x1862022114587648/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766883 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766858.763095:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745500 x1862022114587648/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766883 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766858.763100:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766858.763102:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766858.763104:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114587648:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766858.763107:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114587776:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775766858.763117:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766858.763123:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766858/real 1775766858]  req@ffff9f1485744000 x1862022114587776/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766883 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766858.763130:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744000 x1862022114587776/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766883 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766858.763135:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766858.763137:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766858.763138:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114587776:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775766858.763141:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114587904:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775766858.763151:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766858.763156:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766858/real 1775766858]  req@ffff9f1485747800 x1862022114587904/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766883 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766858.763168:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747800 x1862022114587904/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766883 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766858.763173:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766858.763174:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766858.763176:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114587904:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775766858.763179:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114588032:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775766858.763188:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766858.763193:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766858/real 1775766858]  req@ffff9f1485745c00 x1862022114588032/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766883 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766858.763200:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745c00 x1862022114588032/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766883 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766858.763205:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766858.763207:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766858.763208:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114588032:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766858.763211:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114588160:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775766858.763220:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766858.763225:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766858/real 1775766858]  req@ffff9f1485745180 x1862022114588160/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766883 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766858.763231:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745180 x1862022114588160/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766883 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766858.763240:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766858.763242:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766858.763244:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114588160:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766858.763248:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114588288:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775766858.763256:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766858.763261:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766858/real 1775766858]  req@ffff9f1485745f80 x1862022114588288/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766883 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766858.763268:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745f80 x1862022114588288/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766883 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766858.763272:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766858.763274:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766858.763276:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114588288:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775766859.018675:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766859.786655:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766860.042722:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766860.810625:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775766861.066718:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766861.834635:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:0.0:1775766862.090664:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766862.858653:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766863.114753:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766863.882626:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775766863.882626:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766863.882637:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775766863.882642:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766863.882653:0:8:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4241
00000100:00080000:3.0:1775766863.882667:0:8:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766863.882675:0:8:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775766863.882711:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747100 x1862022114588416/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766863.882739:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766863.882743:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775766863.882745:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766863.882750:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4241
00000100:00080000:3.0:1775766863.882754:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766863.882760:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766863.882771:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766863.882778:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114588416:192.168.202.110@tcp:250:kworker.0
00000100:00100000:3.0:1775766863.882788:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745f80 x1862022114588544/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766863.882797:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766863.882800:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775766863.882802:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766863.882805:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4241
00000100:00080000:3.0:1775766863.882809:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766863.882811:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766863.882815:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766863.882824:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745180 x1862022114588672/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766863.882830:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766863.882833:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775766863.882835:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766863.882838:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4241
00000100:00080000:3.0:1775766863.882840:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766863.882842:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766863.882845:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766863.882852:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745c00 x1862022114588800/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766863.882858:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766863.882861:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775766863.882862:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766863.882865:0:8:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4241
00000100:00080000:3.0:1775766863.882867:0:8:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766863.882869:0:8:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766863.882872:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766863.882881:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747800 x1862022114588928/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775766863.882888:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114588544:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775766863.882889:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766863.882895:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775766863.882896:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766863.882899:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4241
00000100:00080000:3.0:1775766863.882902:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766863.882905:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766863.882908:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766863.882919:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744000 x1862022114589056/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775766863.882925:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114588672:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775766863.882928:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766863.882932:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775766863.882933:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766863.882937:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4241
00000100:00080000:3.0:1775766863.882940:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766863.882943:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766863.882948:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766863.882951:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114588800:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775766863.882959:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745500 x1862022114589184/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766863.882965:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766863.882968:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775766863.882970:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:2.0:1775766863.882976:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114588928:192.168.202.110@tcp:8:kworker.0
00000100:00080000:3.0:1775766863.882976:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4241
00000100:00080000:3.0:1775766863.882982:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766863.882986:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766863.882989:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766863.882998:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744700 x1862022114589312/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775766863.883005:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114589056:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766863.883023:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114589184:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766863.883042:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114589312:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775766864.138754:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766864.906637:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766865.162711:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766865.930626:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766866.186727:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766866.954642:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000400:00000100:0.0:1775766866.955131:0:2393:0:(lib-socket.c:454:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.202.110:988
00000400:00000100:0.0:1775766866.955177:0:2393:0:(acceptor.c:111:lnet_connect_console_error()) Connection to 192.168.202.110@tcp at host 192.168.202.110 was unreachable: the network or that node may be down, or Lustre may be misconfigured.
00000800:00000100:0.0:1775766866.955196:0:2393:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:0.0:1775766866.955246:0:2393:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:0.0:1775766866.955287:0:2393:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:0.0:1775766866.955298:0:2393:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:0.0:1775766866.955308:0:2393:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:0.0:1775766866.955317:0:2393:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:0.0:1775766866.955325:0:2393:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:0.0:1775766866.955334:0:2393:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000100:00100000:2.0:1775766866.955643:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766863/real 1775766866]  req@ffff9f1485744700 x1862022114589312/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766888 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766866.955677:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744700 x1862022114589312/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766888 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766866.955690:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766866.955699:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766866.955704:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114589312:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775766866.955721:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766863/real 1775766866]  req@ffff9f1485747800 x1862022114588928/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766888 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766866.955735:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747800 x1862022114588928/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766888 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766866.955743:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766866.955747:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766866.955750:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114588928:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775766866.955757:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766863/real 1775766866]  req@ffff9f1485744000 x1862022114589056/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766888 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766866.955770:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744000 x1862022114589056/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766888 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766866.955778:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766866.955781:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766866.955784:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114589056:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766866.955789:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766863/real 1775766866]  req@ffff9f1485745500 x1862022114589184/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766888 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766866.955799:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745500 x1862022114589184/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766888 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766866.955805:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766866.955810:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766866.955813:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114589184:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766866.955818:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766863/real 1775766866]  req@ffff9f1485745f80 x1862022114588544/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766888 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766866.955829:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745f80 x1862022114588544/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766888 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766866.955837:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766866.955839:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766866.955842:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114588544:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766866.955850:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766863/real 1775766866]  req@ffff9f1485745180 x1862022114588672/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766888 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766866.955864:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745180 x1862022114588672/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766888 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766866.955872:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766866.955878:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766866.955882:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114588672:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766866.955889:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766863/real 1775766866]  req@ffff9f1485745c00 x1862022114588800/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766888 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766866.955903:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745c00 x1862022114588800/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766888 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766866.955911:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766866.955914:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766866.955917:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114588800:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775766866.955924:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766863/real 1775766866]  req@ffff9f1485747100 x1862022114588416/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766888 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766866.955935:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747100 x1862022114588416/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766888 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766866.955942:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766866.955945:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766866.955948:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114588416:192.168.202.110@tcp:250:kworker.0
00010000:00010000:2.0:1775766867.210706:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766867.978648:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775766868.234757:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766869.002647:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:2.0:1775766869.002677:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766869.002686:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1775766869.002690:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766869.002700:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4246
00000100:00080000:2.0:1775766869.002710:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766869.002716:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775766869.002750:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0380 x1862022114589440/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766869.002781:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766869.002784:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775766869.002785:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766869.002788:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4246
00000100:00080000:2.0:1775766869.002791:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766869.002793:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766869.002802:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766869.002812:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2d80 x1862022114589568/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766869.002819:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766869.002822:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00100000:3.0:1775766869.002842:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114589440:192.168.202.110@tcp:250:kworker.0
00000100:00080000:2.0:1775766869.002843:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766869.002850:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4246
00000100:00080000:2.0:1775766869.002857:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766869.002861:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766869.002865:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766869.002880:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0000 x1862022114589696/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766869.002888:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766869.002890:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775766869.002892:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766869.002895:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4246
00000100:00080000:2.0:1775766869.002897:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766869.002900:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766869.002903:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775766869.002914:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3b80 x1862022114589824/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766869.002919:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766869.002922:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1775766869.002923:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766869.002926:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4246
00000100:00080000:2.0:1775766869.002936:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766869.002940:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766869.002943:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775766869.002951:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0e00 x1862022114589952/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:3.0:1775766869.002964:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:2.0:1775766869.002971:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766869.002977:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775766869.002980:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766869.002989:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4246
00000100:00080000:2.0:1775766869.002997:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766869.003004:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766869.003012:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766869.003033:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766869/real 1775766869]  req@ffff9f14852d0380 x1862022114589440/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766894 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766869.003036:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1880 x1862022114590080/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766869.003090:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766869.003094:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775766869.003097:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766869.003109:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4246
00000100:00100000:3.0:1775766869.003109:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0380 x1862022114589440/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766894 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766869.003131:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766869.003138:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766869.003151:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00080000:3.0:1775766869.003151:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766869.003167:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766869.003175:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114589440:192.168.202.110@tcp:250:kworker.0
00000100:00100000:2.0:1775766869.003186:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3100 x1862022114590208/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775766869.003190:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114589568:192.168.202.110@tcp:38:kworker.0
00000100:00080000:2.0:1775766869.003215:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766869.003218:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775766869.003220:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766869.003227:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4246
00000100:00080000:2.0:1775766869.003246:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000800:00000100:3.0:1775766869.003258:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:2.0:1775766869.003267:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766869.003276:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766869.003280:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766869/real 1775766869]  req@ffff9f14852d2d80 x1862022114589568/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766894 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766869.003307:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2300 x1862022114590336/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775766869.003310:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2d80 x1862022114589568/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766894 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766869.003395:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766869.003404:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766869.003410:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114589568:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766869.003533:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114589696:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775766869.003597:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766869.003611:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766869/real 1775766869]  req@ffff9f14852d0000 x1862022114589696/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766894 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766869.003625:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0000 x1862022114589696/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766894 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766869.003633:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766869.003637:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766869.003640:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114589696:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766869.003647:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114589824:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775766869.003669:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766869.003680:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766869/real 1775766869]  req@ffff9f14852d3b80 x1862022114589824/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766894 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766869.003698:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3b80 x1862022114589824/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766894 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766869.003706:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766869.003709:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766869.003712:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114589824:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775766869.003720:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114589952:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775766869.003741:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766869.003752:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766869/real 1775766869]  req@ffff9f14852d0e00 x1862022114589952/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766894 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766869.003763:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0e00 x1862022114589952/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766894 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766869.003772:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766869.003776:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766869.003779:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114589952:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775766869.003787:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114590080:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775766869.003809:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766869.003821:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766869/real 1775766869]  req@ffff9f14852d1880 x1862022114590080/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766894 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766869.003834:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1880 x1862022114590080/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766894 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766869.003869:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766869.003873:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766869.003876:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114590080:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766869.003883:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114590208:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775766869.003906:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766869.003918:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766869/real 1775766869]  req@ffff9f14852d3100 x1862022114590208/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766894 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766869.003931:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3100 x1862022114590208/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766894 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766869.003939:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766869.003943:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766869.003946:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114590208:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766869.003953:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114590336:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775766869.003980:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766869.003992:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766869/real 1775766869]  req@ffff9f14852d2300 x1862022114590336/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766894 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766869.004003:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2300 x1862022114590336/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766894 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766869.004011:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766869.004014:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766869.004018:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114590336:192.168.202.110@tcp:8:kworker.0
00010000:00010000:0.0:1775766869.258668:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766870.026666:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766870.282802:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766871.050680:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766871.306747:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766872.074662:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766872.330758:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766873.098671:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766873.354706:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1775766874.122660:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766874.122669:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1775766874.122674:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766874.122684:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4251
00000100:00100000:1.0:1775766874.122694:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:2.0:1775766874.122695:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766874.122707:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775766874.122744:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1c00 x1862022114590464/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766874.122766:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766874.122770:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775766874.122771:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766874.122776:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4251
00000100:00080000:2.0:1775766874.122778:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766874.122781:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775766874.122789:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114590464:192.168.202.110@tcp:250:kworker.0
00000008:00100000:2.0:1775766874.122791:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766874.122815:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3480 x1862022114590592/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766874.122830:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766874.122834:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775766874.122837:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766874.122844:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4251
00000100:00080000:2.0:1775766874.122850:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766874.122855:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766874.122862:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000800:00000100:0.0:1775766874.122870:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766874.122879:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2300 x1862022114590720/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766874.122892:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766874.122897:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775766874.122900:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766874.122905:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4251
00000100:00080000:2.0:1775766874.122911:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766874.122918:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766874.122924:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775766874.122934:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3100 x1862022114590848/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775766874.122939:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766874/real 1775766874]  req@ffff9f14852d1c00 x1862022114590464/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766899 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766874.122945:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766874.122953:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1775766874.122957:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766874.122964:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4251
00000100:00080000:2.0:1775766874.122966:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766874.122968:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766874.122971:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775766874.122975:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1c00 x1862022114590464/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766899 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766874.122981:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1880 x1862022114590976/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766874.123000:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766874.123002:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766874.123005:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775766874.123008:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766874.123011:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:2.0:1775766874.123012:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4251
00000100:00080000:2.0:1775766874.123019:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1775766874.123023:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114590464:192.168.202.110@tcp:250:kworker.0
00000100:00080000:2.0:1775766874.123024:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766874.123031:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766874.123038:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0e00 x1862022114591104/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766874.123044:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766874.123047:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775766874.123050:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775766874.123054:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114590592:192.168.202.110@tcp:38:kworker.0
00000100:00080000:2.0:1775766874.123055:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4251
00000100:00080000:2.0:1775766874.123057:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766874.123059:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766874.123062:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766874.123068:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3b80 x1862022114591232/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766874.123078:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766874.123081:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775766874.123083:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766874.123087:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4251
00000100:00080000:2.0:1775766874.123091:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766874.123095:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766874.123101:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:0.0:1775766874.123104:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766874.123118:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0000 x1862022114591360/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775766874.123126:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766874/real 1775766874]  req@ffff9f14852d3480 x1862022114590592/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766899 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766874.123148:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3480 x1862022114590592/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766899 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766874.123159:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766874.123163:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766874.123168:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114590592:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766874.123176:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114590720:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766874.123209:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766874.123220:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766874/real 1775766874]  req@ffff9f14852d2300 x1862022114590720/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766899 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766874.123233:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2300 x1862022114590720/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766899 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766874.123241:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766874.123244:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766874.123248:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114590720:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766874.123255:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114590848:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766874.123273:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766874.123287:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766874/real 1775766874]  req@ffff9f14852d3100 x1862022114590848/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766899 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766874.123308:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3100 x1862022114590848/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766899 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766874.123316:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766874.123319:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766874.123323:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114590848:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775766874.123330:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114590976:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766874.123348:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766874.123359:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766874/real 1775766874]  req@ffff9f14852d1880 x1862022114590976/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766899 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766874.123370:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1880 x1862022114590976/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766899 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766874.123378:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766874.123382:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766874.123386:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114590976:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775766874.123394:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114591104:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766874.123415:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766874.123429:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766874/real 1775766874]  req@ffff9f14852d0e00 x1862022114591104/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766899 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766874.123442:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0e00 x1862022114591104/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766899 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766874.123458:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766874.123463:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766874.123466:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114591104:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766874.123494:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114591232:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766874.123521:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766874.123542:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766874/real 1775766874]  req@ffff9f14852d3b80 x1862022114591232/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766899 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766874.123592:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3b80 x1862022114591232/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766899 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766874.123603:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766874.123607:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766874.123612:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114591232:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766874.123621:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114591360:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766874.123639:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766874.123648:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766874/real 1775766874]  req@ffff9f14852d0000 x1862022114591360/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766899 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766874.123660:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0000 x1862022114591360/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766899 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766874.123669:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766874.123672:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766874.123676:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114591360:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775766874.378753:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775766875.146699:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775766875.402705:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775766876.170641:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:0.0:1775766876.426667:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775766877.194644:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766877.450706:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775766878.218619:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766878.474719:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775766879.242666:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:2.0:1775766879.242695:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766879.242701:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1775766879.242705:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766879.242713:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4256
00000100:00080000:2.0:1775766879.242724:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766879.242730:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775766879.242766:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2d80 x1862022114591488/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766879.242792:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766879.242797:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775766879.242799:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766879.242804:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4256
00000100:00080000:2.0:1775766879.242808:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766879.242813:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766879.242824:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766879.242839:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0380 x1862022114591616/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775766879.242846:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114591488:192.168.202.110@tcp:250:kworker.0
00000100:00080000:2.0:1775766879.242858:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766879.242862:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775766879.242863:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766879.242869:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4256
00000100:00080000:2.0:1775766879.242874:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766879.242878:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766879.242884:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766879.242897:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1500 x1862022114591744/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766879.242908:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766879.242911:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775766879.242913:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766879.242916:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4256
00000100:00080000:2.0:1775766879.242919:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766879.242922:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000800:00000100:0.0:1775766879.242927:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000008:00100000:2.0:1775766879.242928:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775766879.242939:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1f80 x1862022114591872/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766879.242946:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766879.242949:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1775766879.242950:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766879.242954:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4256
00000100:00080000:2.0:1775766879.242958:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766879.242962:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766879.242966:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775766879.242976:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766879/real 1775766879]  req@ffff9f14852d2d80 x1862022114591488/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766904 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766879.242981:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0a80 x1862022114592000/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766879.242999:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1775766879.243003:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2d80 x1862022114591488/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766904 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766879.243004:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775766879.243009:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766879.243019:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766879.243020:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4256
00000100:00080000:0.0:1775766879.243027:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:2.0:1775766879.243028:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1775766879.243031:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114591488:192.168.202.110@tcp:250:kworker.0
00000100:00080000:2.0:1775766879.243036:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766879.243042:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775766879.243055:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114591616:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766879.243057:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2680 x1862022114592128/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766879.243072:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000800:00000100:0.0:1775766879.243078:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:2.0:1775766879.243078:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775766879.243082:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766879.243089:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4256
00000100:00100000:0.0:1775766879.243091:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766879/real 1775766879]  req@ffff9f14852d0380 x1862022114591616/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766904 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766879.243095:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766879.243104:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775766879.243108:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0380 x1862022114591616/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766904 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:2.0:1775766879.243112:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00080000:0.0:1775766879.243116:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766879.243119:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766879.243122:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114591616:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766879.243144:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1180 x1862022114592256/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775766879.243149:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114591744:192.168.202.110@tcp:38:kworker.0
00000100:00080000:2.0:1775766879.243167:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766879.243171:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775766879.243173:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766879.243179:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4256
00000100:00080000:2.0:1775766879.243184:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000800:00000100:0.0:1775766879.243189:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:2.0:1775766879.243192:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766879.243200:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775766879.243207:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766879/real 1775766879]  req@ffff9f14852d1500 x1862022114591744/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766904 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766879.243216:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2a00 x1862022114592384/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775766879.243223:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1500 x1862022114591744/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766904 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766879.243248:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766879.243253:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766879.243257:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114591744:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766879.243264:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114591872:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766879.243287:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766879.243296:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766879/real 1775766879]  req@ffff9f14852d1f80 x1862022114591872/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766904 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766879.243305:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1f80 x1862022114591872/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766904 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766879.243311:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766879.243313:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766879.243315:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114591872:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775766879.243319:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114592000:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766879.243331:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766879.243336:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766879/real 1775766879]  req@ffff9f14852d0a80 x1862022114592000/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766904 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766879.243349:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0a80 x1862022114592000/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766904 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766879.243354:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766879.243356:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766879.243357:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114592000:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775766879.243370:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114592128:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766879.243380:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766879.243385:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766879/real 1775766879]  req@ffff9f14852d2680 x1862022114592128/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766904 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766879.243392:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2680 x1862022114592128/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766904 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766879.243397:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766879.243399:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766879.243401:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114592128:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766879.243404:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114592256:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766879.243413:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766879.243418:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766879/real 1775766879]  req@ffff9f14852d1180 x1862022114592256/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766904 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766879.243425:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1180 x1862022114592256/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766904 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766879.243435:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766879.243437:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766879.243439:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114592256:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766879.243442:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114592384:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766879.243450:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766879.243455:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766879/real 1775766879]  req@ffff9f14852d2a00 x1862022114592384/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766904 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766879.243462:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2a00 x1862022114592384/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766904 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766879.243466:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766879.243468:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766879.243470:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114592384:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775766879.498781:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775766880.266615:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766880.522687:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775766881.290626:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766881.546730:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775766882.314654:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775766882.570757:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775766883.338680:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:0.0:1775766883.594673:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775766884.362642:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775766884.362692:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766884.362700:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775766884.362705:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766884.362717:0:8:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4261
00000100:00080000:3.0:1775766884.362730:0:8:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766884.362738:0:8:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775766884.362786:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746680 x1862022114592512/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766884.362818:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766884.362822:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775766884.362824:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766884.362829:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4261
00000100:00080000:3.0:1775766884.362833:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766884.362838:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766884.362850:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775766884.362865:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114592512:192.168.202.110@tcp:250:kworker.0
00000100:00100000:3.0:1775766884.362865:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744a80 x1862022114592640/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766884.362883:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766884.362887:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775766884.362890:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766884.362894:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4261
00000100:00080000:3.0:1775766884.362899:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766884.362904:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766884.362910:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766884.362923:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747480 x1862022114592768/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766884.362934:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766884.362937:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775766884.362939:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766884.362943:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4261
00000100:00080000:3.0:1775766884.362948:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766884.362952:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000800:00000100:0.0:1775766884.362957:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000008:00100000:3.0:1775766884.362960:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766884.362975:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745880 x1862022114592896/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766884.362986:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766884.362991:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775766884.362993:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766884.362998:0:8:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4261
00000100:00080000:3.0:1775766884.363002:0:8:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1775766884.363003:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766884/real 1775766884]  req@ffff9f1485746680 x1862022114592512/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766909 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766884.363009:0:8:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766884.363019:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775766884.363027:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746680 x1862022114592512/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766909 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766884.363035:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746300 x1862022114593024/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775766884.363038:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766884.363048:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766884.363056:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114592512:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775766884.363069:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766884.363073:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775766884.363089:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766884.363094:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4261
00000100:00080000:3.0:1775766884.363099:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1775766884.363105:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114592640:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775766884.363106:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766884.363114:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766884.363129:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746a00 x1862022114593152/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766884.363142:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766884.363146:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000800:00000100:0.0:1775766884.363150:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775766884.363151:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766884.363159:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4261
00000100:00080000:3.0:1775766884.363164:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1775766884.363171:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766884/real 1775766884]  req@ffff9f1485744a80 x1862022114592640/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766909 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766884.363171:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766884.363178:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775766884.363195:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744a80 x1862022114592640/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766909 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766884.363195:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744380 x1862022114593280/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775766884.363227:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766884.363229:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766884.363232:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00080000:3.0:1775766884.363237:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00100000:0.0:1775766884.363239:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114592640:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775766884.363241:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766884.363250:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4261
00000100:00100000:0.0:1775766884.363251:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114592768:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775766884.363258:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766884.363263:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766884.363269:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:0.0:1775766884.363275:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766884.363284:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744e00 x1862022114593408/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775766884.363287:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766884/real 1775766884]  req@ffff9f1485747480 x1862022114592768/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766909 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766884.363317:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747480 x1862022114592768/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766909 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766884.363323:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766884.363326:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766884.363328:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114592768:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766884.363334:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114592896:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766884.363347:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766884.363354:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766884/real 1775766884]  req@ffff9f1485745880 x1862022114592896/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766909 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766884.363361:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745880 x1862022114592896/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766909 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766884.363367:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766884.363368:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766884.363370:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114592896:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775766884.363373:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114593024:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766884.363383:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766884.363389:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766884/real 1775766884]  req@ffff9f1485746300 x1862022114593024/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766909 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766884.363396:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746300 x1862022114593024/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766909 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766884.363402:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766884.363403:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766884.363405:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114593024:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775766884.363417:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114593152:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766884.363427:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766884.363432:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766884/real 1775766884]  req@ffff9f1485746a00 x1862022114593152/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766909 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766884.363439:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746a00 x1862022114593152/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766909 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766884.363445:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766884.363447:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766884.363449:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114593152:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766884.363452:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114593280:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766884.363461:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766884.363466:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766884/real 1775766884]  req@ffff9f1485744380 x1862022114593280/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766909 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766884.363474:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744380 x1862022114593280/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766909 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766884.363479:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766884.363481:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766884.363483:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114593280:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766884.363486:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114593408:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766884.363496:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766884.363505:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766884/real 1775766884]  req@ffff9f1485744e00 x1862022114593408/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766909 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766884.363518:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744e00 x1862022114593408/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766909 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766884.363526:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766884.363529:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766884.363533:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114593408:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775766884.618788:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775766885.386726:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766885.642719:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775766886.410627:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766886.666730:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775766887.434682:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766887.690703:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766888.458761:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766888.714736:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1775766889.482640:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1775766889.482642:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775766889.482650:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775766889.482654:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766889.482662:0:8:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4266
00000100:00080000:3.0:1775766889.482675:0:8:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766889.482682:0:8:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775766889.482730:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747b80 x1862022114593536/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766889.482753:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766889.482756:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775766889.482758:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766889.482760:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4266
00000100:00080000:3.0:1775766889.482764:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766889.482768:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766889.482775:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775766889.482781:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114593536:192.168.202.110@tcp:250:kworker.0
00000100:00100000:3.0:1775766889.482789:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744e00 x1862022114593664/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766889.482796:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766889.482802:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775766889.482803:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766889.482806:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4266
00000100:00080000:3.0:1775766889.482808:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766889.482810:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766889.482813:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766889.482823:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744380 x1862022114593792/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766889.482829:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766889.482831:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775766889.482832:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766889.482835:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4266
00000100:00080000:3.0:1775766889.482839:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766889.482841:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766889.482844:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:0.0:1775766889.482851:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766889.482854:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746a00 x1862022114593920/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766889.482861:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766889.482864:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775766889.482866:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766889.482870:0:8:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4266
00000100:00080000:3.0:1775766889.482872:0:8:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766889.482874:0:8:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766889.482878:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766889.482888:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746300 x1862022114594048/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775766889.482894:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766889/real 1775766889]  req@ffff9f1485747b80 x1862022114593536/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766914 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766889.482898:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766889.482906:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775766889.482908:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766889.482915:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4266
00000100:00080000:3.0:1775766889.482916:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766889.482919:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766889.482923:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775766889.482929:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747b80 x1862022114593536/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766914 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766889.482933:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745880 x1862022114594176/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775766889.482952:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766889.482953:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766889.482958:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:0.0:1775766889.482960:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:3.0:1775766889.482960:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775766889.482967:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114593536:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775766889.482967:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4266
00000100:00080000:3.0:1775766889.482972:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766889.482975:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766889.482978:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766889.482985:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747480 x1862022114594304/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775766889.482992:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114593664:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775766889.482993:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766889.483015:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775766889.483018:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766889.483034:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4266
00000100:00080000:3.0:1775766889.483038:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766889.483043:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766889.483049:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:0.0:1775766889.483054:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766889.483066:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744a80 x1862022114594432/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775766889.483069:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766889/real 1775766889]  req@ffff9f1485744e00 x1862022114593664/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766914 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766889.483106:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744e00 x1862022114593664/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766914 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766889.483113:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766889.483117:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766889.483120:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114593664:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766889.483125:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114593792:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766889.483138:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766889.483145:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766889/real 1775766889]  req@ffff9f1485744380 x1862022114593792/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766914 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766889.483153:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744380 x1862022114593792/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766914 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766889.483158:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766889.483160:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766889.483162:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114593792:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766889.483166:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114593920:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766889.483176:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766889.483182:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766889/real 1775766889]  req@ffff9f1485746a00 x1862022114593920/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766914 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766889.483189:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746a00 x1862022114593920/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766914 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766889.483194:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766889.483196:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766889.483198:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114593920:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775766889.483202:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114594048:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766889.483211:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766889.483232:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766889/real 1775766889]  req@ffff9f1485746300 x1862022114594048/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766914 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766889.483251:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746300 x1862022114594048/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766914 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766889.483259:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766889.483262:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766889.483266:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114594048:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775766889.483272:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114594176:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766889.483292:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766889.483304:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766889/real 1775766889]  req@ffff9f1485745880 x1862022114594176/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766914 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766889.483313:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745880 x1862022114594176/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766914 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766889.483318:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766889.483320:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766889.483322:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114594176:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766889.483337:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114594304:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766889.483356:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766889.483367:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766889/real 1775766889]  req@ffff9f1485747480 x1862022114594304/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766914 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766889.483380:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747480 x1862022114594304/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766914 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766889.483394:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766889.483398:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766889.483401:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114594304:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766889.483408:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114594432:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766889.483428:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766889.483438:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766889/real 1775766889]  req@ffff9f1485744a80 x1862022114594432/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766914 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766889.483451:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744a80 x1862022114594432/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766914 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766889.483461:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766889.483465:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766889.483468:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114594432:192.168.202.110@tcp:8:kworker.0
00010000:00010000:3.0:1775766889.738827:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766890.506651:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:0.0:1775766890.762723:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766891.530623:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766891.786767:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766892.554641:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766892.810794:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766893.578663:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766893.834750:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766894.602657:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:1.0:1775766894.602672:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766894.602680:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1775766894.602684:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766894.602695:0:8:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4272
00000100:00080000:1.0:1775766894.602706:0:8:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766894.602713:0:8:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775766894.602747:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ffb80 x1862022114594560/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766894.602776:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766894.602779:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775766894.602780:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766894.602783:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4272
00000100:00080000:1.0:1775766894.602787:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766894.602790:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766894.602798:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775766894.602805:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ffb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114594560:192.168.202.110@tcp:250:kworker.0
00000100:00100000:1.0:1775766894.602810:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fea00 x1862022114594688/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766894.602823:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766894.602826:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775766894.602827:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766894.602830:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4272
00000100:00080000:1.0:1775766894.602832:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766894.602835:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766894.602838:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775766894.602846:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff800 x1862022114594816/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766894.602852:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766894.602854:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775766894.602855:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766894.602859:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4272
00000100:00080000:1.0:1775766894.602861:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766894.602864:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766894.602867:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:0.0:1775766894.602871:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775766894.602878:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdf80 x1862022114594944/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766894.602886:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766894.602888:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1775766894.602889:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766894.602892:0:8:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4272
00000100:00080000:1.0:1775766894.602895:0:8:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766894.602898:0:8:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766894.602901:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775766894.602910:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766894/real 1775766894]  req@ffff9f14ab4ffb80 x1862022114594560/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766919 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775766894.602912:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fce00 x1862022114595072/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766894.602942:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766894.602949:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775766894.602952:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766894.602956:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4272
00000100:00080000:1.0:1775766894.602960:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766894.602963:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775766894.602977:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ffb80 x1862022114594560/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766919 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:1.0:1775766894.602986:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00080000:0.0:1775766894.602988:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766894.602993:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775766894.603000:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fca80 x1862022114595200/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775766894.603001:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ffb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114594560:192.168.202.110@tcp:250:kworker.0
00000100:00080000:1.0:1775766894.603015:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766894.603018:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775766894.603019:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766894.603023:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4272
00000100:00080000:1.0:1775766894.603026:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1775766894.603029:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114594688:192.168.202.110@tcp:38:kworker.0
00000100:00080000:1.0:1775766894.603032:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766894.603040:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775766894.603050:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fed80 x1862022114595328/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1775766894.603063:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:1.0:1775766894.603065:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766894.603068:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775766894.603070:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766894.603074:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4272
00000100:00100000:0.0:1775766894.603077:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766894/real 1775766894]  req@ffff9f14ab4fea00 x1862022114594688/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766919 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766894.603079:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766894.603086:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766894.603093:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775766894.603100:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fea00 x1862022114594688/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766919 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775766894.603105:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdc00 x1862022114595456/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775766894.603118:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766894.603125:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766894.603130:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114594688:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766894.603135:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114594816:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766894.603151:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766894.603162:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766894/real 1775766894]  req@ffff9f14ab4ff800 x1862022114594816/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766919 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766894.603175:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff800 x1862022114594816/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766919 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766894.603190:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766894.603193:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766894.603197:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114594816:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766894.603203:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114594944:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766894.603222:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766894.603231:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766894/real 1775766894]  req@ffff9f14ab4fdf80 x1862022114594944/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766919 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766894.603243:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdf80 x1862022114594944/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766919 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766894.603251:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766894.603255:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766894.603258:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114594944:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775766894.603263:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114595072:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766894.603277:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766894.603286:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766894/real 1775766894]  req@ffff9f14ab4fce00 x1862022114595072/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766919 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766894.603295:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fce00 x1862022114595072/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766919 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766894.603300:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766894.603302:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766894.603305:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114595072:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775766894.603309:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114595200:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766894.603322:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766894.603328:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766894/real 1775766894]  req@ffff9f14ab4fca80 x1862022114595200/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766919 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766894.603335:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fca80 x1862022114595200/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766919 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766894.603341:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766894.603342:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766894.603344:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114595200:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766894.603357:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114595328:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766894.603367:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766894.603373:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766894/real 1775766894]  req@ffff9f14ab4fed80 x1862022114595328/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766919 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766894.603381:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fed80 x1862022114595328/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766919 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766894.603386:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766894.603388:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766894.603389:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114595328:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766894.603401:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114595456:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766894.603412:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766894.603419:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766894/real 1775766894]  req@ffff9f14ab4fdc00 x1862022114595456/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766919 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766894.603427:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdc00 x1862022114595456/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766919 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766894.603432:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766894.603433:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766894.603435:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114595456:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775766894.858722:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766895.626626:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766895.882731:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766896.650647:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775766896.906799:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766897.674647:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:0.0:1775766897.930642:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766898.698703:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766898.954739:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766899.722669:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775766899.722751:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766899.722766:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775766899.722774:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766899.722793:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4277
00000100:00080000:3.0:1775766899.722832:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766899.722857:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775766899.722957:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746680 x1862022114595584/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766899.723012:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766899.723020:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775766899.723024:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766899.723035:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4277
00000100:00080000:3.0:1775766899.723044:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766899.723054:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775766899.723061:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114595584:192.168.202.110@tcp:250:kworker.0
00000008:00100000:3.0:1775766899.723085:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766899.723123:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747100 x1862022114595712/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766899.723148:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766899.723160:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000800:00000100:0.0:1775766899.723161:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775766899.723173:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766899.723184:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4277
00000100:00080000:3.0:1775766899.723194:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766899.723203:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766899.723215:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775766899.723216:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766899/real 1775766899]  req@ffff9f1485746680 x1862022114595584/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766924 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766899.723253:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746680 x1862022114595584/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766924 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766899.723254:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745c00 x1862022114595840/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775766899.723318:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766899.723322:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766899.723326:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114595584:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775766899.723344:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1775766899.723351:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114595712:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775766899.723354:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775766899.723364:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766899.723376:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4277
00000100:00080000:3.0:1775766899.723385:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766899.723394:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000800:00000100:0.0:1775766899.723404:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000008:00100000:3.0:1775766899.723405:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775766899.723427:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766899/real 1775766899]  req@ffff9f1485747100 x1862022114595712/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766924 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766899.723438:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745180 x1862022114595968/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775766899.723451:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747100 x1862022114595712/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766924 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766899.723513:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766899.723513:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766899.723532:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00080000:3.0:1775766899.723535:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00100000:0.0:1775766899.723542:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114595712:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775766899.723544:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775766899.723594:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114595840:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766899.723621:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775766899.723644:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4277
00000100:00100000:0.0:1775766899.723656:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766899/real 1775766899]  req@ffff9f1485745c00 x1862022114595840/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766924 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766899.723661:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766899.723681:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775766899.723686:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745c00 x1862022114595840/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766924 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766899.723704:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000008:00100000:3.0:1775766899.723705:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00080000:0.0:1775766899.723712:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766899.723723:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114595840:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766899.723746:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114595968:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775766899.723748:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745f80 x1862022114596096/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1775766899.723780:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775766899.723782:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766899.723791:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00100000:0.0:1775766899.723793:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766899/real 1775766899]  req@ffff9f1485745180 x1862022114595968/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766924 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766899.723801:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766899.723826:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4277
00000100:00100000:0.0:1775766899.723838:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745180 x1862022114595968/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766924 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766899.723841:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766899.723865:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766899.723867:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00080000:0.0:1775766899.723873:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766899.723878:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114595968:192.168.202.110@tcp:8:kworker.0
00000008:00100000:3.0:1775766899.723883:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775766899.723891:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114596096:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766899.723941:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766899.723942:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745500 x1862022114596224/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775766899.723962:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766899/real 1775766899]  req@ffff9f1485745f80 x1862022114596096/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766924 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766899.723969:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766899.723984:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775766899.723994:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775766899.724003:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745f80 x1862022114596096/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766924 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766899.724010:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4277
00000100:00080000:0.0:1775766899.724031:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766899.724034:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766899.724039:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766899.724049:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114596096:192.168.202.110@tcp:8:kworker.0
00000100:00080000:3.0:1775766899.724055:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766899.724068:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775766899.724069:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114596224:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766899.724109:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744000 x1862022114596352/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1775766899.724111:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766899.724129:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766899/real 1775766899]  req@ffff9f1485745500 x1862022114596224/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766924 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766899.724172:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745500 x1862022114596224/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766924 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766899.724176:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766899.724204:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775766899.724211:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766899.724213:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766899.724219:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766899.724241:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114596224:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775766899.724242:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4277
00000100:00080000:3.0:1775766899.724264:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1775766899.724277:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114596352:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775766899.724278:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766899.724301:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:0.0:1775766899.724310:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766899.724321:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766899/real 1775766899]  req@ffff9f1485744000 x1862022114596352/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766924 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766899.724332:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747800 x1862022114596480/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775766899.724348:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744000 x1862022114596352/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766924 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766899.724400:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766899.724408:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766899.724414:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114596352:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766899.724436:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114596480:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766899.724458:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766899.724466:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766899/real 1775766899]  req@ffff9f1485747800 x1862022114596480/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766924 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766899.724475:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747800 x1862022114596480/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766924 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766899.724481:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766899.724483:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766899.724486:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114596480:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775766899.978807:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766900.746727:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766901.002737:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766901.770646:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766902.026708:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766902.794676:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766903.050758:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766903.818653:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775766904.074740:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766904.842703:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:2.0:1775766904.842756:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766904.842778:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1775766904.842784:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766904.842796:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4282
00000100:00080000:2.0:1775766904.842810:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766904.842819:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775766904.842907:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0700 x1862022114596608/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766904.842947:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766904.842951:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775766904.842954:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766904.842959:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4282
00000100:00080000:2.0:1775766904.842976:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766904.842982:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766904.842996:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775766904.843006:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114596608:192.168.202.110@tcp:250:kworker.0
00000100:00100000:2.0:1775766904.843017:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3800 x1862022114596736/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766904.843032:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766904.843037:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775766904.843039:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766904.843044:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4282
00000100:00080000:2.0:1775766904.843049:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766904.843054:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766904.843059:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766904.843075:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2a00 x1862022114596864/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766904.843088:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766904.843092:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775766904.843094:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000800:00000100:0.0:1775766904.843102:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:2.0:1775766904.843107:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4282
00000100:00080000:2.0:1775766904.843114:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766904.843120:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766904.843127:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775766904.843143:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1180 x1862022114596992/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766904.843166:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766904.843170:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1775766904.843175:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775766904.843177:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766904/real 1775766904]  req@ffff9f14852d0700 x1862022114596608/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766929 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766904.843188:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4282
00000100:00080000:2.0:1775766904.843192:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766904.843196:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766904.843201:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775766904.843223:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2680 x1862022114597120/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775766904.843235:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0700 x1862022114596608/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766929 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766904.843240:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766904.843255:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775766904.843261:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766904.843277:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766904.843277:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4282
00000100:00080000:2.0:1775766904.843284:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766904.843292:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:2.0:1775766904.843298:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775766904.843306:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114596608:192.168.202.110@tcp:250:kworker.0
00000008:00100000:2.0:1775766904.843308:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766904.843335:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0a80 x1862022114597248/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775766904.843350:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114596736:192.168.202.110@tcp:38:kworker.0
00000100:00080000:2.0:1775766904.843365:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766904.843373:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775766904.843378:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766904.843389:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4282
00000800:00000100:0.0:1775766904.843391:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:2.0:1775766904.843405:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766904.843418:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775766904.843422:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766904/real 1775766904]  req@ffff9f14852d3800 x1862022114596736/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766929 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:2.0:1775766904.843434:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775766904.843445:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3800 x1862022114596736/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766929 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766904.843459:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00100000:2.0:1775766904.843467:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1f80 x1862022114597376/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775766904.843468:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766904.843482:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114596736:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766904.843506:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114596864:192.168.202.110@tcp:38:kworker.0
00000100:00080000:2.0:1775766904.843508:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766904.843517:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775766904.843520:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766904.843529:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4282
00000100:00080000:2.0:1775766904.843537:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000800:00000100:0.0:1775766904.843542:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:2.0:1775766904.843549:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766904.843600:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775766904.843616:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766904/real 1775766904]  req@ffff9f14852d2a00 x1862022114596864/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766929 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766904.843621:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1500 x1862022114597504/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775766904.843699:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2a00 x1862022114596864/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766929 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766904.843713:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766904.843718:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766904.843723:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114596864:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766904.843732:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114596992:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766904.843764:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766904.843778:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766904/real 1775766904]  req@ffff9f14852d1180 x1862022114596992/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766929 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766904.843793:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1180 x1862022114596992/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766929 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766904.843802:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766904.843806:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766904.843809:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114596992:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775766904.843816:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114597120:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766904.843834:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766904.843845:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766904/real 1775766904]  req@ffff9f14852d2680 x1862022114597120/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766929 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766904.843885:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2680 x1862022114597120/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766929 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766904.843897:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766904.843901:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766904.843905:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114597120:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775766904.843930:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114597248:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766904.843952:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766904.843965:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766904/real 1775766904]  req@ffff9f14852d0a80 x1862022114597248/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766929 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766904.843980:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0a80 x1862022114597248/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766929 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766904.843990:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766904.843994:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766904.843998:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114597248:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766904.844004:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114597376:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766904.844021:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766904.844031:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766904/real 1775766904]  req@ffff9f14852d1f80 x1862022114597376/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766929 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766904.844045:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1f80 x1862022114597376/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766929 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766904.844055:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766904.844059:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766904.844063:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114597376:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766904.844069:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114597504:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766904.844086:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766904.844096:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766904/real 1775766904]  req@ffff9f14852d1500 x1862022114597504/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766929 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766904.844108:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1500 x1862022114597504/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766929 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766904.844117:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766904.844121:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766904.844125:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114597504:192.168.202.110@tcp:8:kworker.0
00010000:00010000:0.0:1775766905.098747:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766905.866620:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766906.122723:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766906.890694:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766907.146800:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766907.914650:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766908.170784:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766908.938669:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766909.194721:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766909.962630:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775766909.962700:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766909.962706:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775766909.962710:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766909.962719:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4287
00000100:00080000:3.0:1775766909.962729:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766909.962735:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775766909.962777:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744700 x1862022114597632/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766909.962797:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766909.962800:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775766909.962802:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766909.962805:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4287
00000100:00080000:3.0:1775766909.962821:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766909.962825:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766909.962834:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775766909.962851:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114597632:192.168.202.110@tcp:250:kworker.0
00000100:00100000:3.0:1775766909.962853:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747800 x1862022114597760/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766909.962868:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766909.962871:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775766909.962873:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766909.962876:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4287
00000100:00080000:3.0:1775766909.962880:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766909.962883:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766909.962887:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766909.962898:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744000 x1862022114597888/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766909.962905:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766909.962908:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775766909.962909:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766909.962913:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4287
00000100:00080000:3.0:1775766909.962916:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766909.962918:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766909.962922:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766909.962929:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745500 x1862022114598016/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1775766909.962932:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775766909.962940:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766909.962943:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00100000:0.0:1775766909.962990:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766909/real 1775766909]  req@ffff9f1485744700 x1862022114597632/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766934 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766909.963009:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775766909.963014:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744700 x1862022114597632/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766934 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766909.963014:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4287
00000100:00080000:3.0:1775766909.963017:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766909.963023:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766909.963029:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00080000:0.0:1775766909.963038:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00100000:3.0:1775766909.963042:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745f80 x1862022114598144/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775766909.963049:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:3.0:1775766909.963054:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1775766909.963058:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114597632:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775766909.963058:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775766909.963061:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766909.963068:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4287
00000100:00080000:3.0:1775766909.963070:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766909.963073:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766909.963076:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766909.963082:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745180 x1862022114598272/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775766909.963090:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114597760:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766909.963111:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766909.963122:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766909/real 1775766909]  req@ffff9f1485747800 x1862022114597760/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766934 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766909.963150:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747800 x1862022114597760/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766934 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766909.963157:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766909.963159:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766909.963162:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114597760:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766909.963167:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114597888:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766909.963181:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766909.963190:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766909/real 1775766909]  req@ffff9f1485744000 x1862022114597888/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766934 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766909.963190:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766909.963197:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775766909.963201:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766909.963209:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4287
00000100:00100000:0.0:1775766909.963213:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744000 x1862022114597888/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766934 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766909.963213:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766909.963220:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00080000:0.0:1775766909.963231:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766909.963243:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766909.963247:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114597888:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766909.963252:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114598016:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766909.963269:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000008:00100000:3.0:1775766909.963277:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775766909.963284:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766909/real 1775766909]  req@ffff9f1485745500 x1862022114598016/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766934 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766909.963290:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745c00 x1862022114598400/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775766909.963304:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745500 x1862022114598016/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766934 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766909.963304:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766909.963310:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775766909.963313:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766909.963321:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4287
00000100:00080000:0.0:1775766909.963326:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766909.963326:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766909.963332:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00080000:0.0:1775766909.963334:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000008:00100000:3.0:1775766909.963337:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775766909.963340:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114598016:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775766909.963348:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114598144:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775766909.963348:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747100 x1862022114598528/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1775766909.963375:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766909.963388:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766909/real 1775766909]  req@ffff9f1485745f80 x1862022114598144/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766934 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766909.963401:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745f80 x1862022114598144/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766934 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766909.963410:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766909.963413:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766909.963418:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114598144:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775766909.963436:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114598272:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766909.963453:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766909.963460:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766909/real 1775766909]  req@ffff9f1485745180 x1862022114598272/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766934 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766909.963468:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745180 x1862022114598272/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766934 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766909.963474:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766909.963476:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766909.963479:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114598272:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766909.963483:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114598400:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766909.963493:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766909.963498:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766909/real 1775766909]  req@ffff9f1485745c00 x1862022114598400/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766934 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766909.963506:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745c00 x1862022114598400/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766934 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766909.963511:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766909.963512:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766909.963514:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114598400:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766909.963517:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114598528:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766909.963526:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766909.963531:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766909/real 1775766909]  req@ffff9f1485747100 x1862022114598528/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766934 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766909.963539:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747100 x1862022114598528/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766934 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766909.963544:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766909.963545:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766909.963547:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114598528:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775766910.218714:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766910.986767:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775766911.242910:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766912.010677:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:0.0:1775766912.266667:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766913.034697:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766913.290787:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766914.058639:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766914.314690:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766915.082723:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775766915.082780:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766915.082793:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775766915.082800:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766915.082817:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4292
00000100:00080000:3.0:1775766915.082833:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766915.082844:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775766915.082897:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746680 x1862022114598656/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766915.082939:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766915.082945:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775766915.082949:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766915.082959:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4292
00000100:00080000:3.0:1775766915.082968:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766915.082976:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775766915.082987:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114598656:192.168.202.110@tcp:250:kworker.0
00000008:00100000:3.0:1775766915.082991:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766915.083024:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744a80 x1862022114598784/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766915.083045:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766915.083052:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775766915.083056:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766915.083063:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4292
00000800:00000100:0.0:1775766915.083071:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775766915.083086:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766915.083097:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766915.083107:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766915.083128:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747480 x1862022114598912/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775766915.083150:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766915/real 1775766915]  req@ffff9f1485746680 x1862022114598656/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766940 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766915.083150:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766915.083168:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775766915.083175:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766915.083192:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4292
00000100:00080000:3.0:1775766915.083200:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1775766915.083208:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746680 x1862022114598656/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766940 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766915.083213:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766915.083228:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00080000:0.0:1775766915.083235:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766915.083244:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766915.083252:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745880 x1862022114599040/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775766915.083253:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114598656:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775766915.083281:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766915.083287:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775766915.083291:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775766915.083294:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114598784:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775766915.083305:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4292
00000100:00080000:3.0:1775766915.083314:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766915.083319:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766915.083328:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766915.083347:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746300 x1862022114599168/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1775766915.083356:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775766915.083376:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766915.083383:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00100000:0.0:1775766915.083388:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766915/real 1775766915]  req@ffff9f1485744a80 x1862022114598784/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766940 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766915.083390:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766915.083404:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4292
00000100:00080000:3.0:1775766915.083417:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766915.083426:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775766915.083429:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744a80 x1862022114598784/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766940 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:3.0:1775766915.083435:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00080000:0.0:1775766915.083449:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766915.083454:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775766915.083463:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746a00 x1862022114599296/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775766915.083464:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114598784:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766915.083484:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114598912:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775766915.083492:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766915.083497:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775766915.083499:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766915.083505:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4292
00000100:00080000:3.0:1775766915.083512:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766915.083517:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000800:00000100:0.0:1775766915.083524:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000008:00100000:3.0:1775766915.083525:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775766915.083543:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766915/real 1775766915]  req@ffff9f1485747480 x1862022114598912/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766940 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766915.083547:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744380 x1862022114599424/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775766915.083602:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747480 x1862022114598912/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766940 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766915.083614:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766915.083627:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775766915.083628:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775766915.083630:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766915.083639:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00080000:3.0:1775766915.083642:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4292
00000100:00100000:0.0:1775766915.083649:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114598912:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775766915.083652:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766915.083665:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775766915.083670:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114599040:192.168.202.110@tcp:8:kworker.0
00000008:00100000:3.0:1775766915.083674:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766915.083698:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744e00 x1862022114599552/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1775766915.083709:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766915.083733:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766915/real 1775766915]  req@ffff9f1485745880 x1862022114599040/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766940 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766915.083799:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745880 x1862022114599040/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766940 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766915.083810:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766915.083813:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766915.083817:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114599040:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775766915.083839:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114599168:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766915.083897:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766915.083910:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766915/real 1775766915]  req@ffff9f1485746300 x1862022114599168/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766940 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766915.083922:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746300 x1862022114599168/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766940 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766915.083931:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766915.083934:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766915.083938:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114599168:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775766915.083944:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114599296:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766915.083961:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766915.083970:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766915/real 1775766915]  req@ffff9f1485746a00 x1862022114599296/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766940 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766915.083982:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746a00 x1862022114599296/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766940 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766915.083997:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766915.084000:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766915.084004:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114599296:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766915.084009:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114599424:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766915.084024:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766915.084033:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766915/real 1775766915]  req@ffff9f1485744380 x1862022114599424/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766940 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766915.084044:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744380 x1862022114599424/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766940 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766915.084053:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766915.084055:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766915.084058:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114599424:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766915.084063:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114599552:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766915.084078:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766915.084086:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766915/real 1775766915]  req@ffff9f1485744e00 x1862022114599552/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766940 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766915.084097:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744e00 x1862022114599552/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766940 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766915.084105:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766915.084108:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766915.084113:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114599552:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775766915.338937:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766916.106670:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766916.362793:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766917.130681:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766917.386717:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766918.154672:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775766918.410849:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775766919.178810:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:0.0:1775766919.434669:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1775766920.202694:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766920.202726:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775766920.202730:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766920.202740:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4297
00000100:00080000:3.0:1775766920.202750:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766920.202759:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775766920.202773:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:3.0:1775766920.202797:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747b80 x1862022114599680/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766920.202820:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766920.202824:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775766920.202826:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766920.202831:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4297
00000100:00080000:3.0:1775766920.202834:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766920.202838:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766920.202850:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766920.202864:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744e00 x1862022114599808/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775766920.202872:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114599680:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775766920.202875:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766920.202880:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775766920.202881:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766920.202884:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4297
00000100:00080000:3.0:1775766920.202886:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766920.202888:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766920.202892:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766920.202900:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744380 x1862022114599936/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766920.202907:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766920.202909:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775766920.202910:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766920.202913:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4297
00000100:00080000:3.0:1775766920.202916:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766920.202918:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766920.202922:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766920.202928:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746a00 x1862022114600064/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766920.202943:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766920.202944:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775766920.202945:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766920.202948:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4297
00000100:00080000:3.0:1775766920.202951:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000800:00000100:0.0:1775766920.202952:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775766920.202956:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766920.202960:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766920.202969:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746300 x1862022114600192/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766920.202975:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766920.202977:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775766920.202978:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766920.202980:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4297
00000100:00080000:3.0:1775766920.202983:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766920.202987:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766920.202991:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766920.203000:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745880 x1862022114600320/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775766920.203002:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766920/real 1775766920]  req@ffff9f1485747b80 x1862022114599680/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766945 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766920.203014:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766920.203022:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775766920.203026:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766920.203029:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4297
00000100:00080000:3.0:1775766920.203032:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1775766920.203037:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747b80 x1862022114599680/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766945 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766920.203037:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766920.203042:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00080000:0.0:1775766920.203054:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00100000:3.0:1775766920.203054:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747480 x1862022114600448/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775766920.203061:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:3.0:1775766920.203069:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766920.203071:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775766920.203072:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775766920.203076:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114599680:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775766920.203076:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4297
00000100:00080000:3.0:1775766920.203078:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766920.203085:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766920.203088:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766920.203095:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744a80 x1862022114600576/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775766920.203105:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114599808:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766920.203124:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766920.203134:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766920/real 1775766920]  req@ffff9f1485744e00 x1862022114599808/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766945 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766920.203144:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744e00 x1862022114599808/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766945 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766920.203150:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766920.203154:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766920.203158:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114599808:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766920.203165:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114599936:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766920.203187:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766920.203199:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766920/real 1775766920]  req@ffff9f1485744380 x1862022114599936/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766945 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766920.203213:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744380 x1862022114599936/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766945 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766920.203221:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766920.203224:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766920.203226:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114599936:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766920.203231:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114600064:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766920.203247:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766920.203254:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766920/real 1775766920]  req@ffff9f1485746a00 x1862022114600064/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766945 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766920.203271:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746a00 x1862022114600064/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766945 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766920.203277:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766920.203279:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766920.203281:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114600064:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775766920.203285:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114600192:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766920.203296:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766920.203302:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766920/real 1775766920]  req@ffff9f1485746300 x1862022114600192/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766945 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766920.203310:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746300 x1862022114600192/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766945 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766920.203315:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766920.203317:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766920.203319:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114600192:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775766920.203322:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114600320:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766920.203333:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766920.203338:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766920/real 1775766920]  req@ffff9f1485745880 x1862022114600320/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766945 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766920.203346:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745880 x1862022114600320/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766945 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766920.203351:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766920.203353:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766920.203354:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114600320:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766920.203357:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114600448:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766920.203367:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766920.203373:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766920/real 1775766920]  req@ffff9f1485747480 x1862022114600448/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766945 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766920.203380:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747480 x1862022114600448/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766945 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766920.203390:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766920.203393:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766920.203395:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114600448:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766920.203408:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114600576:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766920.203417:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766920.203423:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766920/real 1775766920]  req@ffff9f1485744a80 x1862022114600576/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766945 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766920.203430:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744a80 x1862022114600576/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766945 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766920.203436:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766920.203437:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766920.203439:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114600576:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775766920.458766:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775766921.226668:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766921.482770:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775766922.250710:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766922.506870:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775766923.274621:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766923.530741:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775766924.298651:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766924.554788:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775766925.322691:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:1.0:1775766925.322728:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766925.322735:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1775766925.322739:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766925.322774:0:8:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4302
00000100:00080000:1.0:1775766925.322788:0:8:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766925.322797:0:8:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775766925.322844:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fc380 x1862022114600704/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766925.322869:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766925.322873:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775766925.322874:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766925.322877:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4302
00000100:00080000:1.0:1775766925.322881:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766925.322884:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766925.322895:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775766925.322910:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fd880 x1862022114600832/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775766925.322920:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114600704:192.168.202.110@tcp:250:kworker.0
00000100:00080000:1.0:1775766925.322927:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766925.322934:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775766925.322938:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766925.322945:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4302
00000100:00080000:1.0:1775766925.322951:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766925.322957:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766925.322964:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775766925.322975:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fe300 x1862022114600960/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766925.322983:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766925.322987:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775766925.322990:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000800:00000100:0.0:1775766925.322995:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:1.0:1775766925.322998:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4302
00000100:00080000:1.0:1775766925.323003:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766925.323007:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766925.323014:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775766925.323029:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff100 x1862022114601088/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775766925.323040:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766925/real 1775766925]  req@ffff9f14ab4fc380 x1862022114600704/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766950 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766925.323041:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766925.323051:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1775766925.323055:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766925.323062:0:8:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4302
00000100:00100000:0.0:1775766925.323067:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fc380 x1862022114600704/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766950 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766925.323068:0:8:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766925.323077:0:8:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00080000:0.0:1775766925.323083:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000008:00100000:1.0:1775766925.323084:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00080000:0.0:1775766925.323090:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766925.323094:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114600704:192.168.202.110@tcp:250:kworker.0
00000100:00100000:1.0:1775766925.323099:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdc00 x1862022114601216/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766925.323107:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766925.323110:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775766925.323111:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766925.323114:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4302
00000100:00080000:1.0:1775766925.323116:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766925.323119:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766925.323122:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775766925.323132:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114600832:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775766925.323132:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fed80 x1862022114601344/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766925.323146:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766925.323149:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775766925.323151:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766925.323154:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4302
00000100:00080000:1.0:1775766925.323158:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000800:00000100:0.0:1775766925.323162:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:1.0:1775766925.323166:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766925.323188:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775766925.323199:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fca80 x1862022114601472/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775766925.323206:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766925/real 1775766925]  req@ffff9f14ab4fd880 x1862022114600832/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766950 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766925.323210:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766925.323217:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775766925.323218:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766925.323229:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4302
00000100:00080000:1.0:1775766925.323236:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1775766925.323242:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fd880 x1862022114600832/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766950 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766925.323242:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766925.323246:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00080000:0.0:1775766925.323259:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00100000:1.0:1775766925.323262:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fce00 x1862022114601600/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775766925.323266:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766925.323291:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114600832:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766925.323300:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114600960:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766925.323318:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766925.323326:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766925/real 1775766925]  req@ffff9f14ab4fe300 x1862022114600960/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766950 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766925.323334:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fe300 x1862022114600960/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766950 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766925.323340:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766925.323342:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766925.323344:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114600960:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766925.323347:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114601088:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766925.323377:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766925.323384:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766925/real 1775766925]  req@ffff9f14ab4ff100 x1862022114601088/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766950 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766925.323393:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff100 x1862022114601088/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766950 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766925.323400:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766925.323402:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766925.323404:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114601088:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775766925.323408:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114601216:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766925.323419:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766925.323426:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766925/real 1775766925]  req@ffff9f14ab4fdc00 x1862022114601216/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766950 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766925.323434:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdc00 x1862022114601216/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766950 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766925.323440:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766925.323443:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766925.323445:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114601216:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775766925.323459:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114601344:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766925.323471:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766925.323479:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766925/real 1775766925]  req@ffff9f14ab4fed80 x1862022114601344/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766950 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766925.323487:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fed80 x1862022114601344/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766950 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766925.323499:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766925.323502:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766925.323504:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114601344:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766925.323508:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114601472:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766925.323519:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766925.323525:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766925/real 1775766925]  req@ffff9f14ab4fca80 x1862022114601472/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766950 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766925.323533:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fca80 x1862022114601472/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766950 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766925.323560:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766925.323563:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766925.323565:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114601472:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766925.323568:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114601600:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766925.323578:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766925.323584:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766925/real 1775766925]  req@ffff9f14ab4fce00 x1862022114601600/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766950 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766925.323590:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fce00 x1862022114601600/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766950 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766925.323595:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766925.323597:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766925.323599:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114601600:192.168.202.110@tcp:8:kworker.0
00010000:00010000:3.0:1775766925.578738:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775766926.346651:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:0.0:1775766926.602719:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775766927.370644:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766927.626767:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775766928.394693:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766928.650747:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775766929.418679:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766929.674736:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775766930.442701:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775766930.442710:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766930.442719:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775766930.442724:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766930.442733:0:8:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4307
00000100:00080000:3.0:1775766930.442745:0:8:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766930.442752:0:8:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775766930.442796:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746680 x1862022114601728/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766930.442825:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766930.442828:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775766930.442833:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766930.442837:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4307
00000100:00080000:3.0:1775766930.442840:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766930.442843:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766930.442852:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766930.442862:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747100 x1862022114601856/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775766930.442871:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114601728:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775766930.442872:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766930.442881:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775766930.442883:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766930.442886:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4307
00000100:00080000:3.0:1775766930.442888:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766930.442891:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766930.442894:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766930.442903:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745c00 x1862022114601984/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766930.442910:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766930.442913:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775766930.442915:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766930.442919:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4307
00000100:00080000:3.0:1775766930.442924:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766930.442928:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766930.442934:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766930.442947:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745180 x1862022114602112/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766930.442957:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766930.442961:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775766930.442963:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766930.442968:0:8:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4307
00000100:00080000:3.0:1775766930.442974:0:8:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766930.442980:0:8:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775766930.442986:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114601856:192.168.202.110@tcp:38:kworker.0
00000008:00100000:3.0:1775766930.442986:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766930.443006:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745f80 x1862022114602240/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766930.443015:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766930.443019:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775766930.443021:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766930.443025:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4307
00000100:00080000:3.0:1775766930.443029:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766930.443034:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775766930.443036:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114601984:192.168.202.110@tcp:38:kworker.0
00000008:00100000:3.0:1775766930.443041:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766930.443059:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745500 x1862022114602368/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775766930.443070:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114602112:192.168.202.110@tcp:8:kworker.0
00000100:00080000:3.0:1775766930.443086:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766930.443090:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775766930.443093:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766930.443100:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4307
00000100:00080000:3.0:1775766930.443105:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766930.443109:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766930.443115:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775766930.443116:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114602240:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775766930.443149:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744000 x1862022114602496/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766930.443159:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766930.443163:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775766930.443164:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775766930.443168:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114602368:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775766930.443170:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4307
00000100:00080000:3.0:1775766930.443177:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766930.443181:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766930.443186:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766930.443196:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747800 x1862022114602624/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775766930.443214:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114602496:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766930.443266:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114602624:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775766930.698772:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766931.466745:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766931.722766:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766932.490659:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775766932.746719:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766933.514675:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000400:00000100:2.0:1775766933.514942:0:2391:0:(lib-socket.c:454:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.202.110:988
00000400:00000100:2.0:1775766933.514998:0:2391:0:(acceptor.c:111:lnet_connect_console_error()) Connection to 192.168.202.110@tcp at host 192.168.202.110 was unreachable: the network or that node may be down, or Lustre may be misconfigured.
00000800:00000100:2.0:1775766933.515024:0:2391:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:2.0:1775766933.515083:0:2391:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:2.0:1775766933.515104:0:2391:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:2.0:1775766933.515118:0:2391:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:2.0:1775766933.515140:0:2391:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:2.0:1775766933.515158:0:2391:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000100:00100000:0.0:1775766933.515159:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766930/real 1775766933]  req@ffff9f1485747100 x1862022114601856/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766955 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000800:00000100:2.0:1775766933.515179:0:2391:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000100:00100000:0.0:1775766933.515220:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747100 x1862022114601856/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766955 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000800:00000100:2.0:1775766933.515231:0:2391:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000100:00080000:0.0:1775766933.515256:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766933.515265:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766933.515270:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114601856:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766933.515282:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766930/real 1775766933]  req@ffff9f1485745c00 x1862022114601984/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766955 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766933.515299:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745c00 x1862022114601984/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766955 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766933.515307:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766933.515311:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766933.515314:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114601984:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766933.515323:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766930/real 1775766933]  req@ffff9f1485745180 x1862022114602112/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766955 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766933.515335:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745180 x1862022114602112/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766955 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766933.515343:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766933.515347:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766933.515350:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114602112:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775766933.515359:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766930/real 1775766933]  req@ffff9f1485746680 x1862022114601728/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766955 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766933.515373:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746680 x1862022114601728/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766955 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766933.515382:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766933.515387:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766933.515390:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114601728:192.168.202.110@tcp:250:kworker.0
00000100:00100000:0.0:1775766933.515430:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766930/real 1775766933]  req@ffff9f1485744000 x1862022114602496/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766955 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766933.515442:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744000 x1862022114602496/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766955 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766933.515450:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766933.515455:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766933.515458:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114602496:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766933.515466:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766930/real 1775766933]  req@ffff9f1485747800 x1862022114602624/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766955 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766933.515479:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747800 x1862022114602624/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766955 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766933.515487:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766933.515491:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766933.515495:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114602624:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775766933.515504:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766930/real 1775766933]  req@ffff9f1485745f80 x1862022114602240/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766955 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766933.515517:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745f80 x1862022114602240/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766955 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766933.515526:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766933.515529:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766933.515532:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114602240:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775766933.515540:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766930/real 1775766933]  req@ffff9f1485745500 x1862022114602368/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766955 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766933.515551:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745500 x1862022114602368/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766955 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766933.515606:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766933.515611:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766933.515615:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114602368:192.168.202.110@tcp:38:kworker.0
00010000:00010000:0.0:1775766933.770704:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766934.538636:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766934.794725:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1775766935.562638:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1775766935.562640:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775766935.562649:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775766935.562654:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766935.562664:0:8:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4313
00000100:00080000:3.0:1775766935.562676:0:8:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766935.562683:0:8:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775766935.562720:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744700 x1862022114602752/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766935.562751:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766935.562754:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775766935.562756:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766935.562760:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4313
00000100:00080000:3.0:1775766935.562763:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766935.562766:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766935.562775:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766935.562789:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745500 x1862022114602880/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766935.562801:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766935.562805:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00100000:0.0:1775766935.562808:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114602752:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775766935.562808:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766935.562818:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4313
00000100:00080000:3.0:1775766935.562822:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766935.562827:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766935.562833:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766935.562846:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745f80 x1862022114603008/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766935.562856:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766935.562860:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775766935.562862:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766935.562866:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4313
00000100:00080000:3.0:1775766935.562870:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766935.562875:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766935.562881:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:0.0:1775766935.562891:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766935.562896:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747800 x1862022114603136/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766935.562907:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766935.562911:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775766935.562913:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766935.562918:0:8:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4313
00000100:00080000:3.0:1775766935.562922:0:8:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766935.562926:0:8:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766935.562931:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766935.562943:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744000 x1862022114603264/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766935.562954:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1775766935.562956:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766935/real 1775766935]  req@ffff9f1485744700 x1862022114602752/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766960 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766935.562959:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775766935.562970:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766935.562976:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4313
00000100:00080000:3.0:1775766935.562981:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1775766935.562982:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744700 x1862022114602752/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766960 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766935.562988:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766935.562993:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00080000:0.0:1775766935.563008:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00100000:3.0:1775766935.563012:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746680 x1862022114603392/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775766935.563017:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766935.563026:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114602752:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775766935.563032:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766935.563036:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775766935.563038:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766935.563044:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4313
00000100:00080000:3.0:1775766935.563048:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766935.563053:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766935.563061:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775766935.563062:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114602880:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766935.563075:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745180 x1862022114603520/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766935.563088:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766935.563092:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775766935.563094:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000800:00000100:0.0:1775766935.563098:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775766935.563104:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4313
00000100:00080000:3.0:1775766935.563111:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1775766935.563118:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766935/real 1775766935]  req@ffff9f1485745500 x1862022114602880/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766960 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766935.563119:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766935.563129:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775766935.563138:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745500 x1862022114602880/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766960 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766935.563145:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745c00 x1862022114603648/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775766935.563152:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766935.563161:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766935.563167:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114602880:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766935.563175:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114603008:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766935.563198:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766935.563210:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766935/real 1775766935]  req@ffff9f1485745f80 x1862022114603008/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766960 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766935.563224:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745f80 x1862022114603008/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766960 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766935.563232:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766935.563236:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766935.563239:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114603008:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766935.563245:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114603136:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766935.563268:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766935.563279:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766935/real 1775766935]  req@ffff9f1485747800 x1862022114603136/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766960 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766935.563290:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747800 x1862022114603136/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766960 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766935.563305:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766935.563308:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766935.563311:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114603136:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775766935.563317:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114603264:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766935.563338:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766935.563350:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766935/real 1775766935]  req@ffff9f1485744000 x1862022114603264/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766960 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766935.563360:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744000 x1862022114603264/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766960 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766935.563367:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766935.563370:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766935.563373:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114603264:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775766935.563379:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114603392:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766935.563398:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766935.563408:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766935/real 1775766935]  req@ffff9f1485746680 x1862022114603392/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766960 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766935.563419:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746680 x1862022114603392/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766960 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766935.563427:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766935.563429:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766935.563433:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114603392:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766935.563461:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114603520:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766935.563483:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766935.563493:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766935/real 1775766935]  req@ffff9f1485745180 x1862022114603520/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766960 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766935.563505:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745180 x1862022114603520/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766960 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766935.563512:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766935.563515:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766935.563518:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114603520:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766935.563525:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114603648:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766935.563586:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766935.563599:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766935/real 1775766935]  req@ffff9f1485745c00 x1862022114603648/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766960 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766935.563612:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745c00 x1862022114603648/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766960 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766935.563621:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766935.563625:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766935.563629:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114603648:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775766935.818711:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766936.586686:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766936.842773:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766937.610650:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766937.866734:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766938.634665:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766938.890737:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766939.658668:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775766939.914706:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766940.682670:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:2.0:1775766940.682672:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766940.682680:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1775766940.682684:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766940.682693:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4318
00000100:00080000:2.0:1775766940.682704:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766940.682710:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775766940.682745:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0380 x1862022114603776/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766940.682768:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766940.682770:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775766940.682772:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766940.682775:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4318
00000100:00080000:2.0:1775766940.682777:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766940.682780:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766940.682788:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766940.682798:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2d80 x1862022114603904/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775766940.682804:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114603776:192.168.202.110@tcp:250:kworker.0
00000100:00080000:2.0:1775766940.682815:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766940.682819:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775766940.682821:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766940.682827:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4318
00000100:00080000:2.0:1775766940.682831:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766940.682836:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766940.682842:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766940.682855:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0000 x1862022114604032/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766940.682883:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766940.682886:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775766940.682888:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000800:00000100:0.0:1775766940.682896:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:2.0:1775766940.682896:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4318
00000100:00080000:2.0:1775766940.682905:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766940.682923:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766940.682931:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775766940.682941:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766940/real 1775766940]  req@ffff9f14852d0380 x1862022114603776/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766965 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766940.682949:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3b80 x1862022114604160/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766940.682962:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1775766940.682966:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0380 x1862022114603776/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766965 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766940.682967:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1775766940.682971:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766940.682997:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4318
00000100:00080000:2.0:1775766940.683005:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766940.683010:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766940.683010:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766940.683021:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00080000:0.0:1775766940.683022:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766940.683030:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114603776:192.168.202.110@tcp:250:kworker.0
00000100:00100000:2.0:1775766940.683037:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0e00 x1862022114604288/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766940.683047:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766940.683049:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775766940.683051:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766940.683055:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4318
00000100:00100000:0.0:1775766940.683058:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114603904:192.168.202.110@tcp:38:kworker.0
00000100:00080000:2.0:1775766940.683060:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766940.683069:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766940.683074:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766940.683085:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1880 x1862022114604416/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766940.683095:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000800:00000100:0.0:1775766940.683100:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:2.0:1775766940.683101:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775766940.683105:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766940.683110:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4318
00000100:00080000:2.0:1775766940.683114:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766940.683120:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775766940.683121:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766940/real 1775766940]  req@ffff9f14852d2d80 x1862022114603904/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766965 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:2.0:1775766940.683124:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766940.683140:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3100 x1862022114604544/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775766940.683144:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2d80 x1862022114603904/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766965 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766940.683168:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766940.683170:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766940.683177:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775766940.683178:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00080000:2.0:1775766940.683183:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775766940.683184:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114603904:192.168.202.110@tcp:38:kworker.0
00000100:00080000:2.0:1775766940.683197:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4318
00000100:00100000:0.0:1775766940.683200:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114604032:192.168.202.110@tcp:38:kworker.0
00000100:00080000:2.0:1775766940.683222:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766940.683231:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766940.683238:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:0.0:1775766940.683243:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766940.683256:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2300 x1862022114604672/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775766940.683257:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766940/real 1775766940]  req@ffff9f14852d0000 x1862022114604032/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766965 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766940.683269:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0000 x1862022114604032/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766965 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766940.683312:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766940.683316:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766940.683320:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114604032:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766940.683327:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114604160:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766940.683345:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766940.683352:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766940/real 1775766940]  req@ffff9f14852d3b80 x1862022114604160/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766965 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766940.683374:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3b80 x1862022114604160/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766965 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766940.683381:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766940.683383:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766940.683385:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114604160:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775766940.683390:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114604288:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766940.683427:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766940.683433:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766940/real 1775766940]  req@ffff9f14852d0e00 x1862022114604288/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766965 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766940.683446:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0e00 x1862022114604288/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766965 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766940.683451:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766940.683453:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766940.683455:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114604288:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775766940.683468:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114604416:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766940.683479:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766940.683508:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766940/real 1775766940]  req@ffff9f14852d1880 x1862022114604416/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766965 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766940.683518:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1880 x1862022114604416/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766965 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766940.683523:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766940.683525:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766940.683527:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114604416:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766940.683532:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114604544:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766940.683547:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766940.683571:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766940/real 1775766940]  req@ffff9f14852d3100 x1862022114604544/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766965 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766940.683580:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3100 x1862022114604544/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766965 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766940.683590:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766940.683592:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766940.683594:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114604544:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766940.683598:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114604672:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766940.683610:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766940.683617:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766940/real 1775766940]  req@ffff9f14852d2300 x1862022114604672/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766965 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766940.683624:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2300 x1862022114604672/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766965 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766940.683629:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766940.683631:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766940.683633:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114604672:192.168.202.110@tcp:8:kworker.0
00010000:00010000:0.0:1775766940.938682:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766941.706713:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766941.962780:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766942.730725:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766942.986783:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766943.754743:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766944.010714:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766944.778670:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766945.034733:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1775766945.802633:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766945.802642:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1775766945.802647:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766945.802656:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4323
00000100:00080000:2.0:1775766945.802668:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766945.802674:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775766945.802691:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:2.0:1775766945.802711:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3480 x1862022114604800/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766945.802743:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766945.802747:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775766945.802749:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766945.802754:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4323
00000100:00080000:2.0:1775766945.802758:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766945.802765:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766945.802775:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775766945.802786:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114604800:192.168.202.110@tcp:250:kworker.0
00000100:00100000:2.0:1775766945.802796:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1c00 x1862022114604928/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766945.802807:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766945.802810:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775766945.802813:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766945.802817:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4323
00000100:00080000:2.0:1775766945.802824:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766945.802827:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766945.802831:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766945.802850:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2300 x1862022114605056/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766945.802860:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766945.802865:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775766945.802869:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766945.802875:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4323
00000100:00080000:2.0:1775766945.802882:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000800:00000100:1.0:1775766945.802883:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:2.0:1775766945.802895:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766945.802904:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775766945.802922:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3100 x1862022114605184/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766945.802937:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766945.802941:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1775766945.802944:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:1.0:1775766945.802956:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766945/real 1775766945]  req@ffff9f14852d3480 x1862022114604800/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766970 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766945.802957:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4323
00000100:00080000:2.0:1775766945.802972:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766945.802985:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766945.803000:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775766945.803019:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1880 x1862022114605312/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:1.0:1775766945.803020:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3480 x1862022114604800/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766970 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766945.803085:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766945.803089:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775766945.803092:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766945.803097:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766945.803101:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:2.0:1775766945.803107:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4323
00000100:00100000:1.0:1775766945.803113:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114604800:192.168.202.110@tcp:250:kworker.0
00000100:00080000:2.0:1775766945.803115:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766945.803134:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766945.803141:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766945.803157:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0e00 x1862022114605440/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:1.0:1775766945.803162:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114604928:192.168.202.110@tcp:38:kworker.0
00000100:00080000:2.0:1775766945.803185:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766945.803190:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775766945.803191:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766945.803196:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4323
00000100:00080000:2.0:1775766945.803199:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766945.803206:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000800:00000100:1.0:1775766945.803208:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000008:00100000:2.0:1775766945.803221:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775766945.803233:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766945/real 1775766945]  req@ffff9f14852d1c00 x1862022114604928/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766970 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766945.803235:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3b80 x1862022114605568/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:1.0:1775766945.803325:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1c00 x1862022114604928/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766970 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766945.803325:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766945.803356:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766945.803361:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775766945.803365:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00080000:2.0:1775766945.803367:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:1.0:1775766945.803370:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114604928:192.168.202.110@tcp:38:kworker.0
00000100:00080000:2.0:1775766945.803385:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4323
00000100:00100000:1.0:1775766945.803386:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114605056:192.168.202.110@tcp:38:kworker.0
00000100:00080000:2.0:1775766945.803400:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766945.803406:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766945.803415:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:1.0:1775766945.803420:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775766945.803456:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766945/real 1775766945]  req@ffff9f14852d2300 x1862022114605056/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766970 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775766945.803469:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2300 x1862022114605056/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766970 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766945.803495:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775766945.803505:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766945.803507:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0000 x1862022114605696/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:1.0:1775766945.803512:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114605056:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775766945.803534:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114605184:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775766945.803617:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775766945.803634:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766945/real 1775766945]  req@ffff9f14852d3100 x1862022114605184/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766970 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775766945.803644:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3100 x1862022114605184/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766970 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766945.803652:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775766945.803655:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775766945.803658:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114605184:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775766945.803663:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114605312:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775766945.803677:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775766945.803684:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766945/real 1775766945]  req@ffff9f14852d1880 x1862022114605312/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766970 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775766945.803694:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1880 x1862022114605312/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766970 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766945.803700:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775766945.803702:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775766945.803706:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114605312:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775766945.803724:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114605440:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775766945.803737:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775766945.803745:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766945/real 1775766945]  req@ffff9f14852d0e00 x1862022114605440/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766970 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775766945.803755:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0e00 x1862022114605440/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766970 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766945.803762:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775766945.803764:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775766945.803766:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114605440:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775766945.803771:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114605568:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775766945.803784:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775766945.803791:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766945/real 1775766945]  req@ffff9f14852d3b80 x1862022114605568/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766970 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775766945.803801:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3b80 x1862022114605568/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766970 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766945.803807:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775766945.803809:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775766945.803812:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114605568:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775766945.803816:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114605696:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775766945.803828:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775766945.803836:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766945/real 1775766945]  req@ffff9f14852d0000 x1862022114605696/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766970 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775766945.803845:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0000 x1862022114605696/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766970 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766945.803852:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775766945.803854:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775766945.803856:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114605696:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775766946.058861:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766946.826705:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775766947.082734:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766947.850655:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:0.0:1775766948.106750:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766948.874771:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766949.130721:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766949.898677:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766950.154724:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766950.922684:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775766950.922735:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766950.922743:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775766950.922748:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766950.922758:0:8:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4328
00000100:00080000:3.0:1775766950.922770:0:8:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766950.922778:0:8:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775766950.922830:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747100 x1862022114605824/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766950.922864:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766950.922868:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775766950.922870:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766950.922876:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4328
00000100:00080000:3.0:1775766950.922880:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766950.922884:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766950.922897:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766950.922927:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744a80 x1862022114605952/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:1.0:1775766950.922935:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114605824:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775766950.922940:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766950.922944:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775766950.922946:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766950.922951:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4328
00000100:00080000:3.0:1775766950.922955:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766950.922959:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766950.922964:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775766950.922989:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747480 x1862022114606080/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766950.922998:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766950.923001:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775766950.923002:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766950.923006:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4328
00000100:00080000:3.0:1775766950.923008:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766950.923012:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766950.923016:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:1.0:1775766950.923023:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775766950.923025:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745880 x1862022114606208/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766950.923033:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766950.923035:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775766950.923037:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766950.923040:0:8:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4328
00000100:00080000:3.0:1775766950.923043:0:8:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766950.923046:0:8:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766950.923051:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775766950.923060:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746300 x1862022114606336/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:1.0:1775766950.923069:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766950/real 1775766950]  req@ffff9f1485747100 x1862022114605824/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766975 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775766950.923069:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766950.923073:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775766950.923075:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766950.923080:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4328
00000100:00080000:3.0:1775766950.923083:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766950.923087:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775766950.923088:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747100 x1862022114605824/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766975 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:3.0:1775766950.923092:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00080000:1.0:1775766950.923099:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00100000:3.0:1775766950.923103:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746a00 x1862022114606464/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766950.923104:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775766950.923108:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114605824:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775766950.923112:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766950.923115:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775766950.923117:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766950.923120:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4328
00000100:00080000:3.0:1775766950.923123:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766950.923126:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775766950.923132:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775766950.923133:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114605952:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775766950.923140:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744380 x1862022114606592/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775766950.923150:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775766950.923153:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000800:00000100:1.0:1775766950.923155:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775766950.923155:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775766950.923160:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4328
00000100:00080000:3.0:1775766950.923163:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775766950.923166:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775766950.923167:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766950/real 1775766950]  req@ffff9f1485744a80 x1862022114605952/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766975 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:3.0:1775766950.923171:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775766950.923180:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744a80 x1862022114605952/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766975 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775766950.923180:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744e00 x1862022114606720/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766950.923196:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775766950.923199:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775766950.923202:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114605952:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775766950.923209:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114606080:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775766950.923229:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775766950.923240:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766950/real 1775766950]  req@ffff9f1485747480 x1862022114606080/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766975 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775766950.923248:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747480 x1862022114606080/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766975 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766950.923254:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775766950.923256:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775766950.923258:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114606080:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775766950.923268:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114606208:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775766950.923295:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775766950.923301:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766950/real 1775766950]  req@ffff9f1485745880 x1862022114606208/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766975 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775766950.923309:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745880 x1862022114606208/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766975 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766950.923314:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775766950.923316:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775766950.923320:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114606208:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775766950.923323:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114606336:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775766950.923333:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775766950.923338:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766950/real 1775766950]  req@ffff9f1485746300 x1862022114606336/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766975 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775766950.923345:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746300 x1862022114606336/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766975 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766950.923351:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775766950.923353:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775766950.923354:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114606336:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775766950.923358:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114606464:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775766950.923367:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775766950.923372:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766950/real 1775766950]  req@ffff9f1485746a00 x1862022114606464/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766975 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775766950.923398:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746a00 x1862022114606464/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766975 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766950.923416:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775766950.923418:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775766950.923420:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114606464:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775766950.923434:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114606592:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775766950.923448:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775766950.923455:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766950/real 1775766950]  req@ffff9f1485744380 x1862022114606592/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766975 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775766950.923463:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744380 x1862022114606592/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766975 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766950.923478:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775766950.923480:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775766950.923483:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114606592:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775766950.923486:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114606720:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775766950.923495:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775766950.923500:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766950/real 1775766950]  req@ffff9f1485744e00 x1862022114606720/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766975 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775766950.923507:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744e00 x1862022114606720/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766975 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766950.923512:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775766950.923514:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775766950.923516:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114606720:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775766951.178803:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766951.946678:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766952.202772:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775766952.970772:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766953.226817:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775766953.994648:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775766954.250742:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775766955.018666:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:0.0:1775766955.274745:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775766956.042662:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:0.0:1775766956.042685:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766956.042691:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1775766956.042696:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766956.042705:0:8:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4333
00000100:00080000:0.0:1775766956.042718:0:8:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766956.042730:0:8:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775766956.042787:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3480 x1862022114606848/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775766956.042811:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766956.042814:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775766956.042815:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766956.042819:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4333
00000100:00080000:0.0:1775766956.042821:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766956.042824:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775766956.042833:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775766956.042842:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d1180 x1862022114606976/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775766956.042850:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766956.042852:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:0.0:1775766956.042854:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:1.0:1775766956.042858:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114606848:192.168.202.110@tcp:250:kworker.0
00000100:00080000:0.0:1775766956.042859:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4333
00000100:00080000:0.0:1775766956.042864:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766956.042869:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775766956.042873:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775766956.042880:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0e00 x1862022114607104/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775766956.042885:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766956.042888:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775766956.042889:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766956.042892:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4333
00000100:00080000:0.0:1775766956.042894:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766956.042896:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775766956.042899:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775766956.042909:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d1f80 x1862022114607232/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775766956.042914:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766956.042916:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1775766956.042917:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766956.042919:0:8:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4333
00000100:00080000:0.0:1775766956.042929:0:8:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766956.042932:0:8:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775766956.042936:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775766956.042945:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0380 x1862022114607360/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:1.0:1775766956.042946:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:0.0:1775766956.042954:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766956.042957:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775766956.042959:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766956.042962:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4333
00000100:00080000:0.0:1775766956.042964:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766956.042968:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775766956.042970:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775766956.042977:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d2a00 x1862022114607488/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775766956.042982:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766956.042984:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:0.0:1775766956.042985:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766956.042988:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4333
00000100:00080000:0.0:1775766956.042990:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766956.042993:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775766956.042996:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775766956.043002:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0a80 x1862022114607616/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775766956.043008:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766956.043011:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775766956.043013:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:1.0:1775766956.043014:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766956/real 1775766956]  req@ffff9f14a04d3480 x1862022114606848/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766981 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766956.043018:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4333
00000100:00080000:0.0:1775766956.043022:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766956.043027:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775766956.043034:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775766956.043044:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3100 x1862022114607744/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:1.0:1775766956.043063:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3480 x1862022114606848/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766981 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766956.043077:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775766956.043083:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775766956.043088:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114606848:192.168.202.110@tcp:250:kworker.0
00000100:00100000:1.0:1775766956.043116:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114606976:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775766956.043143:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775766956.043158:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766956/real 1775766956]  req@ffff9f14a04d1180 x1862022114606976/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766981 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775766956.043173:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1180 x1862022114606976/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766981 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766956.043183:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775766956.043187:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775766956.043191:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114606976:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775766956.043198:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114607104:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775766956.043218:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775766956.043236:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766956/real 1775766956]  req@ffff9f14a04d0e00 x1862022114607104/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766981 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775766956.043250:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0e00 x1862022114607104/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766981 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766956.043276:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775766956.043280:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775766956.043285:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114607104:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775766956.043291:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114607232:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775766956.043314:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775766956.043342:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766956/real 1775766956]  req@ffff9f14a04d1f80 x1862022114607232/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766981 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775766956.043357:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1f80 x1862022114607232/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766981 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766956.043366:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775766956.043380:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775766956.043384:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114607232:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775766956.043391:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114607360:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775766956.043414:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775766956.043426:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766956/real 1775766956]  req@ffff9f14a04d0380 x1862022114607360/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766981 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775766956.043440:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0380 x1862022114607360/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766981 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766956.043450:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775766956.043453:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775766956.043458:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114607360:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775766956.043464:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114607488:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775766956.043485:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775766956.043497:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766956/real 1775766956]  req@ffff9f14a04d2a00 x1862022114607488/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766981 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775766956.043522:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d2a00 x1862022114607488/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766981 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766956.043531:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775766956.043534:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775766956.043538:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114607488:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775766956.043544:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114607616:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775766956.043590:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775766956.043602:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766956/real 1775766956]  req@ffff9f14a04d0a80 x1862022114607616/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766981 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775766956.043615:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0a80 x1862022114607616/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766981 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766956.043624:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775766956.043627:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775766956.043630:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114607616:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775766956.043637:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114607744:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775766956.043658:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775766956.043669:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766956/real 1775766956]  req@ffff9f14a04d3100 x1862022114607744/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766981 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775766956.043681:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3100 x1862022114607744/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766981 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766956.043690:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775766956.043693:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775766956.043696:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114607744:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775766956.298722:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775766957.066640:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766957.322701:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775766958.090628:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766958.346824:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775766959.114674:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766959.370783:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775766960.138643:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766960.394733:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1775766961.162653:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766961.162665:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00100000:1.0:1775766961.162668:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:0.0:1775766961.162670:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766961.162685:0:8:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4338
00000100:00080000:0.0:1775766961.162699:0:8:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766961.162708:0:8:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775766961.162762:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3b80 x1862022114607872/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775766961.162788:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766961.162791:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775766961.162793:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766961.162796:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4338
00000100:00080000:0.0:1775766961.162800:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766961.162804:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775766961.162814:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775766961.162824:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d2300 x1862022114608000/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775766961.162833:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766961.162837:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00100000:1.0:1775766961.162837:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114607872:192.168.202.110@tcp:250:kworker.0
00000100:00080000:0.0:1775766961.162839:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766961.162846:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4338
00000100:00080000:0.0:1775766961.162851:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766961.162854:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775766961.162858:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775766961.162868:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0000 x1862022114608128/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775766961.162875:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766961.162879:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775766961.162880:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766961.162883:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4338
00000100:00080000:0.0:1775766961.162885:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766961.162888:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775766961.162891:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775766961.162899:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d1500 x1862022114608256/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775766961.162905:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766961.162908:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1775766961.162910:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766961.162912:0:8:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4338
00000100:00080000:0.0:1775766961.162915:0:8:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766961.162918:0:8:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775766961.162922:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:1.0:1775766961.162926:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766961.162934:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3800 x1862022114608384/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775766961.162942:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766961.162945:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775766961.162947:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766961.162951:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4338
00000100:00080000:0.0:1775766961.162954:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766961.162957:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775766961.162960:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775766961.162969:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3100 x1862022114608512/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775766961.162976:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766961.162979:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:0.0:1775766961.162981:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:1.0:1775766961.162982:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766961/real 1775766961]  req@ffff9f14a04d3b80 x1862022114607872/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766986 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766961.162986:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4338
00000100:00080000:0.0:1775766961.162991:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766961.162996:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775766961.163004:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775766961.163020:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0a80 x1862022114608640/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:1.0:1775766961.163028:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3b80 x1862022114607872/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766986 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766961.163029:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766961.163036:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775766961.163039:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766961.163047:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4338
00000100:00080000:0.0:1775766961.163054:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766961.163058:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00080000:1.0:1775766961.163061:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000008:00100000:0.0:1775766961.163063:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00080000:1.0:1775766961.163074:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766961.163078:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d2a00 x1862022114608768/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:1.0:1775766961.163084:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114607872:192.168.202.110@tcp:250:kworker.0
00000100:00100000:1.0:1775766961.163121:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114608000:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775766961.163153:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775766961.163170:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766961/real 1775766961]  req@ffff9f14a04d2300 x1862022114608000/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766986 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775766961.163187:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d2300 x1862022114608000/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766986 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766961.163198:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775766961.163202:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775766961.163206:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114608000:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775766961.163214:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114608128:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775766961.163239:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775766961.163252:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766961/real 1775766961]  req@ffff9f14a04d0000 x1862022114608128/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766986 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775766961.163268:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0000 x1862022114608128/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766986 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766961.163296:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775766961.163300:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775766961.163304:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114608128:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775766961.163312:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114608256:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775766961.163339:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775766961.163351:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766961/real 1775766961]  req@ffff9f14a04d1500 x1862022114608256/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766986 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775766961.163366:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1500 x1862022114608256/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766986 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766961.163376:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775766961.163380:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775766961.163384:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114608256:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775766961.163391:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114608384:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775766961.163409:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775766961.163420:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766961/real 1775766961]  req@ffff9f14a04d3800 x1862022114608384/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766986 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775766961.163434:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3800 x1862022114608384/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766986 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766961.163444:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775766961.163448:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775766961.163451:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114608384:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775766961.163459:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114608512:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775766961.163495:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775766961.163509:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766961/real 1775766961]  req@ffff9f14a04d3100 x1862022114608512/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766986 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775766961.163525:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3100 x1862022114608512/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766986 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766961.163535:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775766961.163538:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775766961.163543:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114608512:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775766961.163551:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114608640:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775766961.163603:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775766961.163617:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766961/real 1775766961]  req@ffff9f14a04d0a80 x1862022114608640/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766986 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775766961.163631:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0a80 x1862022114608640/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766986 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766961.163641:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775766961.163645:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775766961.163649:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114608640:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775766961.163657:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114608768:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775766961.163682:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775766961.163695:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766961/real 1775766961]  req@ffff9f14a04d2a00 x1862022114608768/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766986 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775766961.163710:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d2a00 x1862022114608768/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766986 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766961.163721:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775766961.163725:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775766961.163730:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114608768:192.168.202.110@tcp:8:kworker.0
00010000:00010000:3.0:1775766961.418745:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775766962.186648:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:0.0:1775766962.442720:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775766963.210651:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766963.466705:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766964.234703:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766964.490695:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775766965.258676:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766965.514712:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1775766966.282682:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766966.282690:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1775766966.282693:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766966.282702:0:8:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4343
00000100:00080000:0.0:1775766966.282712:0:8:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766966.282719:0:8:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775766966.282761:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:0.0:1775766966.282778:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0380 x1862022114608896/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775766966.282802:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766966.282806:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775766966.282808:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766966.282814:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4343
00000100:00080000:0.0:1775766966.282819:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766966.282824:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775766966.282829:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114608896:192.168.202.110@tcp:250:kworker.0
00000008:00100000:0.0:1775766966.282836:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775766966.282854:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d1f80 x1862022114609024/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775766966.282868:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766966.282874:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:0.0:1775766966.282876:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766966.282881:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4343
00000100:00080000:0.0:1775766966.282886:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766966.282891:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000800:00000100:1.0:1775766966.282896:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000008:00100000:0.0:1775766966.282898:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775766966.282913:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0e00 x1862022114609152/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775766966.282925:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766966.282930:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775766966.282932:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:1.0:1775766966.282937:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766966/real 1775766966]  req@ffff9f14a04d0380 x1862022114608896/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766991 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766966.282939:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4343
00000100:00080000:0.0:1775766966.282947:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766966.282954:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775766966.282961:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775766966.282961:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0380 x1862022114608896/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766991 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766966.282974:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775766966.282979:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766966.282983:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d1180 x1862022114609280/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:1.0:1775766966.282986:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114608896:192.168.202.110@tcp:250:kworker.0
00000100:00080000:0.0:1775766966.283003:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766966.283007:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1775766966.283009:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:1.0:1775766966.283010:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114609024:192.168.202.110@tcp:38:kworker.0
00000100:00080000:0.0:1775766966.283019:0:8:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4343
00000100:00080000:0.0:1775766966.283025:0:8:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766966.283030:0:8:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000800:00000100:1.0:1775766966.283035:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000008:00100000:0.0:1775766966.283040:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775766966.283048:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766966/real 1775766966]  req@ffff9f14a04d1f80 x1862022114609024/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766991 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766966.283056:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3480 x1862022114609408/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:1.0:1775766966.283062:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1f80 x1862022114609024/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766991 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766966.283072:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766966.283075:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766966.283076:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775766966.283080:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766966.283082:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00080000:0.0:1775766966.283087:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4343
00000100:00100000:1.0:1775766966.283088:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114609024:192.168.202.110@tcp:38:kworker.0
00000100:00080000:0.0:1775766966.283093:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:1.0:1775766966.283097:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114609152:192.168.202.110@tcp:38:kworker.0
00000100:00080000:0.0:1775766966.283100:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775766966.283106:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775766966.283117:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0380 x1862022114609536/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:1.0:1775766966.283121:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:0.0:1775766966.283130:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:1.0:1775766966.283134:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766966/real 1775766966]  req@ffff9f14a04d0e00 x1862022114609152/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766991 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766966.283135:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:0.0:1775766966.283138:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766966.283145:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4343
00000100:00080000:0.0:1775766966.283151:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766966.283156:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775766966.283161:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775766966.283167:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0e00 x1862022114609152/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766991 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766966.283170:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d2a00 x1862022114609664/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766966.283215:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766966.283216:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766966.283231:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775766966.283236:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766966.283240:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00080000:0.0:1775766966.283243:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4343
00000100:00100000:1.0:1775766966.283246:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114609152:192.168.202.110@tcp:38:kworker.0
00000100:00080000:0.0:1775766966.283248:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766966.283272:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775766966.283276:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114609280:192.168.202.110@tcp:8:kworker.0
00000008:00100000:0.0:1775766966.283278:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775766966.283292:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0a80 x1862022114609792/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:1.0:1775766966.283302:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775766966.283314:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766966/real 1775766966]  req@ffff9f14a04d1180 x1862022114609280/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766991 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775766966.283326:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1180 x1862022114609280/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766991 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766966.283334:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775766966.283337:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775766966.283339:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114609280:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775766966.283353:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114609408:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775766966.283364:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775766966.283370:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766966/real 1775766966]  req@ffff9f14a04d3480 x1862022114609408/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766991 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775766966.283378:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3480 x1862022114609408/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766991 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766966.283388:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775766966.283390:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775766966.283392:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114609408:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775766966.283396:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114609536:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775766966.283405:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775766966.283411:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766966/real 1775766966]  req@ffff9f14a04d0380 x1862022114609536/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766991 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775766966.283418:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0380 x1862022114609536/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766991 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766966.283424:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775766966.283426:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775766966.283428:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114609536:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775766966.283431:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114609664:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775766966.283441:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775766966.283446:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766966/real 1775766966]  req@ffff9f14a04d2a00 x1862022114609664/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766991 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775766966.283454:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d2a00 x1862022114609664/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766991 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766966.283459:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775766966.283461:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775766966.283463:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114609664:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775766966.283483:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114609792:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775766966.283494:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775766966.283499:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766966/real 1775766966]  req@ffff9f14a04d0a80 x1862022114609792/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766991 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775766966.283506:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0a80 x1862022114609792/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766991 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766966.283511:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775766966.283513:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775766966.283514:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114609792:192.168.202.110@tcp:8:kworker.0
00010000:00010000:1.0:1775766966.538734:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775766967.306678:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766967.562748:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775766968.330663:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775766968.586736:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775766969.354623:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:0.0:1775766969.610650:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775766970.378704:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766970.634775:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775766971.402696:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:2.0:1775766971.402702:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766971.402715:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1775766971.402721:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766971.402733:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4348
00000100:00080000:2.0:1775766971.402745:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766971.402752:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775766971.402794:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2d80 x1862022114609920/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766971.402826:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766971.402830:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775766971.402833:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766971.402839:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4348
00000100:00080000:2.0:1775766971.402845:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766971.402851:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766971.402863:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775766971.402867:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114609920:192.168.202.110@tcp:250:kworker.0
00000100:00100000:2.0:1775766971.402896:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0380 x1862022114610048/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766971.402910:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766971.402914:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775766971.402917:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766971.402922:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4348
00000100:00080000:2.0:1775766971.402927:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766971.402934:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766971.402942:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000800:00000100:1.0:1775766971.402946:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766971.402965:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1500 x1862022114610176/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766971.402980:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766971.402984:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775766971.402988:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766971.402995:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4348
00000100:00100000:1.0:1775766971.403000:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766971/real 1775766971]  req@ffff9f14852d2d80 x1862022114609920/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766996 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766971.403001:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766971.403017:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766971.403031:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775766971.403037:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2d80 x1862022114609920/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775766996 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766971.403048:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00100000:2.0:1775766971.403055:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1f80 x1862022114610304/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766971.403057:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775766971.403088:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114609920:192.168.202.110@tcp:250:kworker.0
00000100:00080000:2.0:1775766971.403091:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766971.403097:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1775766971.403109:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:1.0:1775766971.403118:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114610048:192.168.202.110@tcp:38:kworker.0
00000100:00080000:2.0:1775766971.403119:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4348
00000100:00080000:2.0:1775766971.403144:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766971.403152:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766971.403159:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:1.0:1775766971.403180:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775766971.403195:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766971/real 1775766971]  req@ffff9f14852d0380 x1862022114610048/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766996 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766971.403206:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0a80 x1862022114610432/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:1.0:1775766971.403208:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0380 x1862022114610048/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766996 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766971.403257:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775766971.403267:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775766971.403275:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114610048:192.168.202.110@tcp:38:kworker.0
00000100:00080000:2.0:1775766971.403278:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:1.0:1775766971.403283:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114610176:192.168.202.110@tcp:38:kworker.0
00000100:00080000:2.0:1775766971.403287:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775766971.403291:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766971.403299:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4348
00000100:00080000:2.0:1775766971.403305:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000800:00000100:1.0:1775766971.403306:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:2.0:1775766971.403320:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775766971.403324:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766971/real 1775766971]  req@ffff9f14852d1500 x1862022114610176/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766996 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:2.0:1775766971.403338:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775766971.403343:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1500 x1862022114610176/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766996 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766971.403350:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775766971.403353:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775766971.403356:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114610176:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775766971.403363:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2680 x1862022114610560/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:1.0:1775766971.403367:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114610304:192.168.202.110@tcp:8:kworker.0
00000100:00080000:2.0:1775766971.403391:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766971.403394:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775766971.403397:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766971.403404:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4348
00000800:00000100:1.0:1775766971.403410:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:2.0:1775766971.403413:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766971.403420:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766971.403426:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775766971.403430:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766971/real 1775766971]  req@ffff9f14852d1f80 x1862022114610304/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766996 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766971.403444:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1180 x1862022114610688/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:1.0:1775766971.403450:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1f80 x1862022114610304/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766996 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766971.403476:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775766971.403479:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775766971.403482:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114610304:192.168.202.110@tcp:8:kworker.0
00000100:00080000:2.0:1775766971.403507:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766971.403512:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775766971.403515:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:1.0:1775766971.403518:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114610432:192.168.202.110@tcp:8:kworker.0
00000100:00080000:2.0:1775766971.403526:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4348
00000100:00080000:2.0:1775766971.403531:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766971.403536:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766971.403543:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:1.0:1775766971.403545:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775766971.403583:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766971/real 1775766971]  req@ffff9f14852d0a80 x1862022114610432/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766996 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775766971.403593:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0a80 x1862022114610432/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766996 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766971.403601:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775766971.403608:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775766971.403615:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2a00 x1862022114610816/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:1.0:1775766971.403617:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114610432:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775766971.403634:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114610560:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775766971.403662:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775766971.403677:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766971/real 1775766971]  req@ffff9f14852d2680 x1862022114610560/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766996 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775766971.403693:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2680 x1862022114610560/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766996 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766971.403699:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775766971.403702:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775766971.403704:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114610560:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775766971.403708:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114610688:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775766971.403719:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775766971.403724:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766971/real 1775766971]  req@ffff9f14852d1180 x1862022114610688/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766996 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775766971.403732:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1180 x1862022114610688/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775766996 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766971.403737:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775766971.403739:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775766971.403741:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114610688:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775766971.403751:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114610816:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775766971.403761:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775766971.403767:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766971/real 1775766971]  req@ffff9f14852d2a00 x1862022114610816/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766996 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775766971.403774:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2a00 x1862022114610816/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775766996 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766971.403784:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775766971.403786:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775766971.403789:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114610816:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775766971.658708:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775766972.426717:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766972.682778:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775766973.450630:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775766973.706736:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775766974.474668:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766974.730736:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775766975.498687:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775766975.754743:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1775766976.522690:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766976.522702:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00100000:1.0:1775766976.522703:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:2.0:1775766976.522708:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766976.522720:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4354
00000100:00080000:2.0:1775766976.522732:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766976.522740:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775766976.522794:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3800 x1862022114610944/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766976.522823:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766976.522826:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775766976.522827:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766976.522831:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4354
00000100:00080000:2.0:1775766976.522834:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766976.522836:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766976.522846:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766976.522857:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0700 x1862022114611072/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766976.522868:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:1.0:1775766976.522873:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114610944:192.168.202.110@tcp:250:kworker.0
00000100:00080000:2.0:1775766976.522874:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775766976.522880:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766976.522894:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4354
00000100:00080000:2.0:1775766976.522897:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766976.522900:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766976.522903:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766976.522918:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2a00 x1862022114611200/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766976.522926:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766976.522928:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775766976.522930:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766976.522933:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4354
00000100:00080000:2.0:1775766976.522935:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766976.522938:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766976.522943:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775766976.522951:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1180 x1862022114611328/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766976.522957:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766976.522960:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1775766976.522962:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766976.522964:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4354
00000100:00080000:2.0:1775766976.522968:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766976.522974:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000800:00000100:1.0:1775766976.522982:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000008:00100000:2.0:1775766976.522988:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775766976.523008:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2680 x1862022114611456/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766976.523024:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766976.523028:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775766976.523030:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:1.0:1775766976.523033:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766976/real 1775766976]  req@ffff9f14852d3800 x1862022114610944/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767001 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766976.523041:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4354
00000100:00080000:2.0:1775766976.523054:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766976.523062:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766976.523070:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775766976.523071:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3800 x1862022114610944/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767001 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766976.523088:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0a80 x1862022114611584/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766976.523089:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775766976.523108:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775766976.523142:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114610944:192.168.202.110@tcp:250:kworker.0
00000100:00080000:2.0:1775766976.523152:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766976.523157:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775766976.523160:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766976.523167:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4354
00000100:00080000:2.0:1775766976.523172:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766976.523177:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775766976.523184:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114611072:192.168.202.110@tcp:38:kworker.0
00000008:00100000:2.0:1775766976.523186:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766976.523209:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1f80 x1862022114611712/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766976.523222:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000800:00000100:1.0:1775766976.523231:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:2.0:1775766976.523234:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775766976.523237:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766976.523244:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4354
00000100:00080000:2.0:1775766976.523250:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766976.523256:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775766976.523257:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766976/real 1775766976]  req@ffff9f14852d0700 x1862022114611072/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767001 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:2.0:1775766976.523268:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775766976.523288:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1500 x1862022114611840/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:1.0:1775766976.523306:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0700 x1862022114611072/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767001 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766976.523321:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775766976.523325:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775766976.523328:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114611072:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775766976.523340:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114611200:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775766976.523357:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775766976.523365:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766976/real 1775766976]  req@ffff9f14852d2a00 x1862022114611200/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767001 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775766976.523376:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2a00 x1862022114611200/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767001 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766976.523382:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775766976.523385:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775766976.523388:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114611200:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775766976.523394:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114611328:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775766976.523407:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775766976.523414:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766976/real 1775766976]  req@ffff9f14852d1180 x1862022114611328/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767001 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775766976.523424:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1180 x1862022114611328/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767001 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766976.523437:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775766976.523440:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775766976.523442:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114611328:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775766976.523447:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114611456:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775766976.523459:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775766976.523466:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766976/real 1775766976]  req@ffff9f14852d2680 x1862022114611456/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767001 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775766976.523475:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2680 x1862022114611456/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767001 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766976.523482:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775766976.523484:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775766976.523487:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114611456:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775766976.523491:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114611584:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775766976.523503:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775766976.523511:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766976/real 1775766976]  req@ffff9f14852d0a80 x1862022114611584/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767001 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775766976.523520:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0a80 x1862022114611584/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767001 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766976.523526:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775766976.523528:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775766976.523532:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114611584:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775766976.523549:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114611712:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775766976.523605:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775766976.523622:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766976/real 1775766976]  req@ffff9f14852d1f80 x1862022114611712/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767001 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775766976.523639:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1f80 x1862022114611712/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767001 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766976.523652:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775766976.523656:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775766976.523661:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114611712:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775766976.523670:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114611840:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775766976.523690:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775766976.523700:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766976/real 1775766976]  req@ffff9f14852d1500 x1862022114611840/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767001 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775766976.523711:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1500 x1862022114611840/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767001 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766976.523718:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775766976.523721:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775766976.523723:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114611840:192.168.202.110@tcp:8:kworker.0
00010000:00010000:0.0:1775766976.778852:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775766977.546626:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766977.802808:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775766978.570667:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766978.826766:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775766979.594629:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766979.850790:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775766980.618668:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775766980.874728:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775766981.642633:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:2.0:1775766981.642663:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766981.642672:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1775766981.642677:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766981.642688:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4359
00000100:00080000:2.0:1775766981.642702:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766981.642711:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775766981.642762:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0380 x1862022114611968/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766981.642795:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766981.642800:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775766981.642802:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766981.642824:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4359
00000100:00080000:2.0:1775766981.642833:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766981.642841:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775766981.642857:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114611968:192.168.202.110@tcp:250:kworker.0
00000008:00100000:2.0:1775766981.642858:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766981.642901:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2d80 x1862022114612096/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766981.642919:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766981.642925:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775766981.642927:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766981.642933:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4359
00000100:00080000:2.0:1775766981.642940:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766981.642946:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766981.642967:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000800:00000100:1.0:1775766981.642976:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775766981.642982:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0000 x1862022114612224/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766981.642994:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766981.642998:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775766981.643000:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766981.643004:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4359
00000100:00080000:2.0:1775766981.643008:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766981.643013:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766981.643020:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775766981.643028:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766981/real 1775766981]  req@ffff9f14852d0380 x1862022114611968/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767006 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766981.643036:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3b80 x1862022114612352/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:1.0:1775766981.643048:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0380 x1862022114611968/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767006 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766981.643068:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766981.643071:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766981.643073:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775766981.643080:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114611968:192.168.202.110@tcp:250:kworker.0
00000100:00080000:2.0:1775766981.643080:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1775766981.643085:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766981.643091:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4359
00000100:00080000:2.0:1775766981.643097:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766981.643102:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775766981.643107:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114612096:192.168.202.110@tcp:38:kworker.0
00000008:00100000:2.0:1775766981.643108:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775766981.643127:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0e00 x1862022114612480/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:1.0:1775766981.643134:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775766981.643148:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766981/real 1775766981]  req@ffff9f14852d2d80 x1862022114612096/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767006 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766981.643148:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766981.643159:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775766981.643164:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:1.0:1775766981.643168:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2d80 x1862022114612096/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767006 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766981.643174:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4359
00000100:00080000:1.0:1775766981.643177:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775766981.643180:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775766981.643182:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114612096:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775766981.643190:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114612224:192.168.202.110@tcp:38:kworker.0
00000100:00080000:2.0:1775766981.643191:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766981.643202:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766981.643209:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000800:00000100:1.0:1775766981.643210:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775766981.643219:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766981/real 1775766981]  req@ffff9f14852d0000 x1862022114612224/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767006 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766981.643227:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1880 x1862022114612608/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:1.0:1775766981.643231:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0000 x1862022114612224/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767006 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766981.643259:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775766981.643263:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766981.643264:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775766981.643270:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114612224:192.168.202.110@tcp:38:kworker.0
00000100:00080000:2.0:1775766981.643272:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775766981.643275:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:1.0:1775766981.643276:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114612352:192.168.202.110@tcp:8:kworker.0
00000100:00080000:2.0:1775766981.643286:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4359
00000100:00080000:2.0:1775766981.643292:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766981.643298:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000800:00000100:1.0:1775766981.643299:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000008:00100000:2.0:1775766981.643307:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775766981.643309:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766981/real 1775766981]  req@ffff9f14852d3b80 x1862022114612352/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767006 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775766981.643319:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3b80 x1862022114612352/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767006 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766981.643325:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3100 x1862022114612736/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766981.643329:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775766981.643335:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775766981.643342:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114612352:192.168.202.110@tcp:8:kworker.0
00000100:00080000:2.0:1775766981.643357:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:1.0:1775766981.643360:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114612480:192.168.202.110@tcp:8:kworker.0
00000100:00080000:2.0:1775766981.643367:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775766981.643370:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766981.643376:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4359
00000800:00000100:1.0:1775766981.643381:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:2.0:1775766981.643382:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766981.643389:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766981.643395:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775766981.643398:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766981/real 1775766981]  req@ffff9f14852d0e00 x1862022114612480/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767006 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775766981.643410:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2300 x1862022114612864/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:1.0:1775766981.644687:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0e00 x1862022114612480/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767006 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766981.644697:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775766981.644701:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775766981.644704:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114612480:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775766981.644714:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114612608:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775766981.644768:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775766981.644786:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766981/real 1775766981]  req@ffff9f14852d1880 x1862022114612608/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767006 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775766981.644794:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1880 x1862022114612608/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767006 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766981.644800:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775766981.644802:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775766981.644804:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114612608:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775766981.644819:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114612736:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775766981.644831:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775766981.644837:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766981/real 1775766981]  req@ffff9f14852d3100 x1862022114612736/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767006 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775766981.644844:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3100 x1862022114612736/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767006 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766981.644853:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775766981.644856:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775766981.644858:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114612736:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775766981.644891:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114612864:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775766981.644902:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775766981.644908:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766981/real 1775766981]  req@ffff9f14852d2300 x1862022114612864/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767006 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775766981.644915:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2300 x1862022114612864/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767006 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766981.644920:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775766981.644922:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775766981.644924:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114612864:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775766981.898821:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775766982.666657:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775766982.922776:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775766983.690612:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:0.0:1775766983.946670:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775766984.714644:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766984.970711:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775766985.738699:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766985.994734:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1775766986.762633:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766986.762644:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1775766986.762649:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766986.762661:0:8:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4364
00000100:00100000:1.0:1775766986.762662:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:2.0:1775766986.762674:0:8:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766986.762682:0:8:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775766986.762741:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1c00 x1862022114612992/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766986.762771:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766986.762775:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775766986.762776:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766986.762781:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4364
00000100:00080000:2.0:1775766986.762783:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766986.762786:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766986.762797:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766986.762806:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3480 x1862022114613120/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766986.762815:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766986.762818:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775766986.762820:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766986.762823:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4364
00000100:00080000:2.0:1775766986.762835:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766986.762838:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766986.762841:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775766986.762849:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114612992:192.168.202.110@tcp:250:kworker.0
00000100:00100000:2.0:1775766986.762854:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2300 x1862022114613248/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766986.762873:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766986.762875:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775766986.762876:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766986.762880:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4364
00000100:00080000:2.0:1775766986.762882:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766986.762885:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766986.762889:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775766986.762910:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3100 x1862022114613376/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766986.762917:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766986.762920:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1775766986.762922:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766986.762925:0:8:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4364
00000100:00080000:2.0:1775766986.762927:0:8:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766986.762929:0:8:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766986.762933:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775766986.762942:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1880 x1862022114613504/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766986.762947:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766986.762949:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775766986.762950:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766986.762953:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4364
00000100:00080000:2.0:1775766986.762955:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766986.762959:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766986.762962:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766986.762970:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0e00 x1862022114613632/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766986.762975:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766986.762987:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000800:00000100:1.0:1775766986.762989:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:2.0:1775766986.762990:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766986.763021:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4364
00000100:00080000:2.0:1775766986.763026:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775766986.763045:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766986.763052:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775766986.763063:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3b80 x1862022114613760/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775766986.763072:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775766986.763075:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775766986.763077:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775766986.763083:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4364
00000100:00080000:2.0:1775766986.763086:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:1.0:1775766986.763090:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766986/real 1775766986]  req@ffff9f14852d1c00 x1862022114612992/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767011 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775766986.763095:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775766986.763106:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775766986.763119:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0000 x1862022114613888/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:1.0:1775766986.763124:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1c00 x1862022114612992/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767011 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766986.763165:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775766986.763171:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775766986.763175:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114612992:192.168.202.110@tcp:250:kworker.0
00000100:00100000:1.0:1775766986.763197:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114613120:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775766986.763217:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775766986.763227:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766986/real 1775766986]  req@ffff9f14852d3480 x1862022114613120/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767011 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775766986.763235:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3480 x1862022114613120/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767011 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766986.763241:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775766986.763243:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775766986.763245:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114613120:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775766986.763250:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114613248:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775766986.763267:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775766986.763276:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766986/real 1775766986]  req@ffff9f14852d2300 x1862022114613248/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767011 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775766986.763289:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2300 x1862022114613248/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767011 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766986.763298:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775766986.763302:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775766986.763305:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114613248:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775766986.763313:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114613376:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775766986.763332:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775766986.763340:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766986/real 1775766986]  req@ffff9f14852d3100 x1862022114613376/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767011 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775766986.763349:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3100 x1862022114613376/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767011 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766986.763354:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775766986.763357:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775766986.763359:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114613376:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775766986.763363:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114613504:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775766986.763373:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775766986.763379:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766986/real 1775766986]  req@ffff9f14852d1880 x1862022114613504/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767011 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775766986.763386:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1880 x1862022114613504/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767011 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766986.763391:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775766986.763393:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775766986.763394:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114613504:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775766986.763399:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114613632:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775766986.763409:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775766986.763415:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766986/real 1775766986]  req@ffff9f14852d0e00 x1862022114613632/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767011 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775766986.763422:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0e00 x1862022114613632/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767011 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766986.763427:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775766986.763429:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775766986.763431:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114613632:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775766986.763434:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114613760:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775766986.763444:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775766986.763449:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766986/real 1775766986]  req@ffff9f14852d3b80 x1862022114613760/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767011 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775766986.763456:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3b80 x1862022114613760/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767011 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766986.763461:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775766986.763462:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775766986.763464:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114613760:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775766986.763468:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114613888:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775766986.763479:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775766986.763487:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766986/real 1775766986]  req@ffff9f14852d0000 x1862022114613888/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767011 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775766986.763499:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0000 x1862022114613888/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767011 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775766986.763514:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775766986.763518:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775766986.763521:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114613888:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775766987.018777:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775766987.786699:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775766988.042655:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775766988.810651:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766989.066700:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775766989.834679:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775766990.090725:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775766990.858635:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:0.0:1775766991.114690:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1775766991.882692:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766991.882700:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1775766991.882704:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00100000:2.0:1775766991.882712:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:1.0:1775766991.882713:0:8:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4369
00000100:00080000:1.0:1775766991.882730:0:8:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766991.882737:0:8:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775766991.882773:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdf80 x1862022114614016/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766991.882799:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766991.882802:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775766991.882803:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766991.882806:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4369
00000100:00080000:1.0:1775766991.882808:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766991.882811:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766991.882819:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775766991.882827:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff800 x1862022114614144/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766991.882835:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766991.882838:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775766991.882839:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766991.882842:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4369
00000100:00080000:1.0:1775766991.882845:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1775766991.882847:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114614016:192.168.202.110@tcp:250:kworker.0
00000100:00080000:1.0:1775766991.882849:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766991.882855:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775766991.882862:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fea00 x1862022114614272/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766991.882869:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766991.882872:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775766991.882873:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766991.882875:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4369
00000100:00080000:1.0:1775766991.882878:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766991.882880:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766991.882883:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775766991.882891:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ffb80 x1862022114614400/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766991.882896:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766991.882898:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1775766991.882900:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766991.882902:0:8:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4369
00000100:00080000:1.0:1775766991.882906:0:8:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766991.882908:0:8:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766991.882911:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:0.0:1775766991.882917:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775766991.882920:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fce00 x1862022114614528/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766991.882927:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766991.882930:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775766991.882931:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766991.882934:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4369
00000100:00080000:1.0:1775766991.882937:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766991.882940:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766991.882943:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775766991.882974:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fca80 x1862022114614656/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766991.882981:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766991.882983:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775766991.882984:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766991.882988:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4369
00000100:00080000:1.0:1775766991.882990:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766991.882992:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775766991.882996:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775766991.883004:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766991/real 1775766991]  req@ffff9f14ab4fdf80 x1862022114614016/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767016 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775766991.883006:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fed80 x1862022114614784/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775766991.883037:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775766991.883039:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775766991.883040:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775766991.883044:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4369
00000100:00080000:1.0:1775766991.883046:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775766991.883050:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775766991.883051:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdf80 x1862022114614016/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767016 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:1.0:1775766991.883053:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775766991.883063:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdc00 x1862022114614912/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775766991.883072:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766991.883080:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766991.883086:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114614016:192.168.202.110@tcp:250:kworker.0
00000100:00100000:0.0:1775766991.883095:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114614144:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766991.883122:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766991.883136:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766991/real 1775766991]  req@ffff9f14ab4ff800 x1862022114614144/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767016 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766991.883156:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff800 x1862022114614144/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767016 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766991.883166:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766991.883169:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766991.883173:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114614144:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766991.883198:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114614272:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766991.883220:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766991.883231:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766991/real 1775766991]  req@ffff9f14ab4fea00 x1862022114614272/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767016 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766991.883245:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fea00 x1862022114614272/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767016 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766991.883252:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766991.883256:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766991.883259:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114614272:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766991.883266:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ffb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114614400:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766991.883286:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766991.883297:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766991/real 1775766991]  req@ffff9f14ab4ffb80 x1862022114614400/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767016 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766991.883310:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ffb80 x1862022114614400/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767016 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766991.883325:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766991.883329:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766991.883332:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ffb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114614400:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775766991.883346:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114614528:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766991.883366:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766991.883376:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766991/real 1775766991]  req@ffff9f14ab4fce00 x1862022114614528/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767016 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766991.883389:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fce00 x1862022114614528/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767016 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766991.883409:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766991.883412:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766991.883416:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114614528:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775766991.883423:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114614656:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766991.883445:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766991.883456:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766991/real 1775766991]  req@ffff9f14ab4fca80 x1862022114614656/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767016 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766991.883469:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fca80 x1862022114614656/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767016 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766991.883477:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766991.883480:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766991.883484:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114614656:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766991.883490:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114614784:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775766991.883512:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766991.883522:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766991/real 1775766991]  req@ffff9f14ab4fed80 x1862022114614784/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767016 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766991.883534:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fed80 x1862022114614784/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767016 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766991.883542:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766991.883545:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766991.883549:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114614784:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766991.883585:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114614912:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775766991.883606:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775766991.883627:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766991/real 1775766991]  req@ffff9f14ab4fdc00 x1862022114614912/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767016 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775766991.883639:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdc00 x1862022114614912/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767016 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775766991.883648:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775766991.883651:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775766991.883655:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114614912:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775766992.138771:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766992.906656:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766993.162746:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766993.930735:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766994.186782:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766994.954680:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775766995.210762:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766995.978675:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766996.234774:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1775766997.002684:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766997.002692:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1775766997.002698:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766997.002708:0:8:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4374
00000100:00080000:0.0:1775766997.002720:0:8:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00100000:2.0:1775766997.002725:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:0.0:1775766997.002727:0:8:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775766997.002766:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3100 x1862022114615040/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775766997.002795:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766997.002798:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775766997.002800:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766997.002804:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4374
00000100:00080000:0.0:1775766997.002806:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766997.002809:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775766997.002818:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775766997.002829:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3800 x1862022114615168/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775766997.002840:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766997.002844:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:0.0:1775766997.002845:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766997.002849:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4374
00000100:00080000:0.0:1775766997.002853:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00100000:1.0:1775766997.002854:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114615040:192.168.202.110@tcp:250:kworker.0
00000100:00080000:0.0:1775766997.002857:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775766997.002864:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775766997.002873:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d1500 x1862022114615296/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775766997.002886:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766997.002889:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775766997.002890:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766997.002893:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4374
00000100:00080000:0.0:1775766997.002912:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766997.002916:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775766997.002921:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775766997.002931:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0000 x1862022114615424/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775766997.002938:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766997.002941:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1775766997.002943:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766997.002946:0:8:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4374
00000100:00080000:0.0:1775766997.002949:0:8:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766997.002951:0:8:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775766997.002955:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775766997.002963:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d2300 x1862022114615552/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775766997.002971:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766997.002974:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775766997.002975:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766997.002978:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4374
00000100:00080000:0.0:1775766997.002981:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766997.002985:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775766997.002990:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775766997.003000:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3b80 x1862022114615680/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:1.0:1775766997.003010:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114615168:192.168.202.110@tcp:38:kworker.0
00000100:00080000:0.0:1775766997.003015:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766997.003020:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:0.0:1775766997.003022:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766997.003029:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4374
00000100:00080000:0.0:1775766997.003033:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766997.003039:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775766997.003047:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775766997.003066:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114615296:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775766997.003072:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0a80 x1862022114615808/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775766997.003082:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775766997.003085:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775766997.003088:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775766997.003104:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4374
00000100:00080000:0.0:1775766997.003110:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775766997.003114:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775766997.003119:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775766997.003121:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114615424:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775766997.003132:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d2a00 x1862022114615936/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:1.0:1775766997.003179:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114615552:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775766997.003222:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114615680:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775766997.003252:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114615808:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775766997.003285:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114615936:192.168.202.110@tcp:8:kworker.0
00010000:00010000:3.0:1775766997.258758:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766998.026819:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:0.0:1775766998.282777:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775766999.050754:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775766999.306723:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767000.074960:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000400:00000100:1.0:1775767000.075152:0:2390:0:(lib-socket.c:454:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.202.110:988
00000400:00000100:1.0:1775767000.075237:0:2390:0:(acceptor.c:111:lnet_connect_console_error()) Connection to 192.168.202.110@tcp at host 192.168.202.110 was unreachable: the network or that node may be down, or Lustre may be misconfigured.
00000800:00000100:1.0:1775767000.075256:0:2390:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:1.0:1775767000.075312:0:2390:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:1.0:1775767000.075321:0:2390:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:1.0:1775767000.075330:0:2390:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:1.0:1775767000.075339:0:2390:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:1.0:1775767000.075353:0:2390:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:1.0:1775767000.075370:0:2390:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:1.0:1775767000.075389:0:2390:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000100:00100000:0.0:1775767000.075417:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766997/real 1775767000]  req@ffff9f14a04d3b80 x1862022114615680/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767022 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767000.075462:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3b80 x1862022114615680/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767022 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767000.075479:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767000.075488:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767000.075495:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114615680:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767000.075507:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766997/real 1775767000]  req@ffff9f14a04d0a80 x1862022114615808/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767022 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767000.075541:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0a80 x1862022114615808/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767022 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767000.075551:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767000.075595:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767000.075600:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114615808:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767000.075613:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766997/real 1775767000]  req@ffff9f14a04d2a00 x1862022114615936/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767022 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767000.075631:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d2a00 x1862022114615936/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767022 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767000.075641:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767000.075645:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767000.075650:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114615936:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775767000.075659:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766997/real 1775767000]  req@ffff9f14a04d3800 x1862022114615168/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767022 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767000.075691:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3800 x1862022114615168/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767022 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767000.075701:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767000.075706:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767000.075710:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114615168:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767000.075720:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766997/real 1775767000]  req@ffff9f14a04d1500 x1862022114615296/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767022 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767000.075737:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1500 x1862022114615296/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767022 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767000.075747:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767000.075752:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767000.075756:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114615296:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767000.075768:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766997/real 1775767000]  req@ffff9f14a04d0000 x1862022114615424/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767022 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767000.075783:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0000 x1862022114615424/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767022 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767000.075793:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767000.075798:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767000.075804:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114615424:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775767000.075814:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766997/real 1775767000]  req@ffff9f14a04d2300 x1862022114615552/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767022 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767000.075833:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d2300 x1862022114615552/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767022 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767000.075853:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767000.075858:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767000.075884:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114615552:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775767000.075896:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775766997/real 1775767000]  req@ffff9f14a04d3100 x1862022114615040/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767022 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767000.075912:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3100 x1862022114615040/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767022 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767000.075922:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767000.075926:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767000.075931:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114615040:192.168.202.110@tcp:250:kworker.0
00010000:00010000:2.0:1775767000.330761:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767001.098770:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767001.354720:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1775767002.122690:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767002.122699:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1775767002.122704:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767002.122715:0:8:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4379
00000100:00080000:1.0:1775767002.122726:0:8:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767002.122732:0:8:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775767002.122772:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:1.0:1775767002.122784:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff100 x1862022114616064/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767002.122808:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767002.122811:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775767002.122812:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767002.122816:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4379
00000100:00080000:1.0:1775767002.122819:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767002.122823:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767002.122831:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767002.122840:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114616064:192.168.202.110@tcp:250:kworker.0
00000100:00100000:1.0:1775767002.122858:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fe300 x1862022114616192/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767002.122869:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767002.122872:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775767002.122874:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767002.122877:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4379
00000100:00080000:1.0:1775767002.122880:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767002.122883:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767002.122887:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767002.122896:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fd880 x1862022114616320/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767002.122903:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767002.122906:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775767002.122907:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767002.122909:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4379
00000100:00080000:1.0:1775767002.122911:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767002.122914:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767002.122917:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:0.0:1775767002.122921:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767002.122925:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fc380 x1862022114616448/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767002.122932:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767002.122935:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1775767002.122936:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767002.122939:0:8:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4379
00000100:00080000:1.0:1775767002.122943:0:8:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767002.122945:0:8:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767002.122948:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775767002.122955:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdc00 x1862022114616576/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767002.122961:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767002.122963:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775767002.122964:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767002.122971:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4379
00000100:00100000:0.0:1775767002.122972:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767002/real 1775767002]  req@ffff9f14ab4ff100 x1862022114616064/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767027 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767002.122973:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767002.122979:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767002.122984:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767002.122995:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fed80 x1862022114616704/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767002.123001:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767002.123004:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775767002.123005:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775767002.123008:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff100 x1862022114616064/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767027 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767002.123011:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4379
00000100:00080000:1.0:1775767002.123016:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767002.123021:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767002.123028:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767002.123037:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fca80 x1862022114616832/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767002.123049:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767002.123057:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767002.123061:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:1.0:1775767002.123061:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775767002.123065:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767002.123071:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4379
00000100:00100000:0.0:1775767002.123073:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114616064:192.168.202.110@tcp:250:kworker.0
00000100:00080000:1.0:1775767002.123074:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767002.123080:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767002.123084:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775767002.123091:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fce00 x1862022114616960/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775767002.123109:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114616192:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767002.123137:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767002.123149:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767002/real 1775767002]  req@ffff9f14ab4fe300 x1862022114616192/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767027 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767002.123168:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fe300 x1862022114616192/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767027 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767002.123178:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767002.123182:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767002.123185:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114616192:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767002.123192:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114616320:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767002.123213:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767002.123226:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767002/real 1775767002]  req@ffff9f14ab4fd880 x1862022114616320/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767027 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767002.123238:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fd880 x1862022114616320/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767027 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767002.123245:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767002.123248:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767002.123251:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114616320:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767002.123258:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114616448:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767002.123278:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767002.123289:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767002/real 1775767002]  req@ffff9f14ab4fc380 x1862022114616448/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767027 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767002.123302:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fc380 x1862022114616448/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767027 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767002.123310:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767002.123313:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767002.123317:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114616448:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775767002.123322:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114616576:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767002.123342:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767002.123351:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767002/real 1775767002]  req@ffff9f14ab4fdc00 x1862022114616576/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767027 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767002.123361:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdc00 x1862022114616576/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767027 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767002.123368:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767002.123371:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767002.123373:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114616576:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775767002.123379:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114616704:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767002.123398:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767002.123410:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767002/real 1775767002]  req@ffff9f14ab4fed80 x1862022114616704/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767027 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767002.123423:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fed80 x1862022114616704/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767027 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767002.123430:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767002.123433:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767002.123436:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114616704:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767002.123441:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114616832:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767002.123462:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767002.123473:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767002/real 1775767002]  req@ffff9f14ab4fca80 x1862022114616832/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767027 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767002.123484:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fca80 x1862022114616832/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767027 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767002.123492:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767002.123495:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767002.123498:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114616832:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767002.123504:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114616960:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767002.123524:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767002.123535:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767002/real 1775767002]  req@ffff9f14ab4fce00 x1862022114616960/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767027 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767002.123548:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fce00 x1862022114616960/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767027 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767002.123585:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767002.123589:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767002.123592:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114616960:192.168.202.110@tcp:8:kworker.0
00010000:00010000:1.0:1775767002.378719:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767003.146864:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767003.402707:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767004.170618:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767004.426735:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767005.194693:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:0.0:1775767005.450672:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767006.218616:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767006.474740:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767007.242697:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775767007.242719:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767007.242727:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775767007.242732:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767007.242745:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4384
00000100:00080000:3.0:1775767007.242756:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767007.242761:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775767007.242823:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747b80 x1862022114617088/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767007.242847:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767007.242851:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775767007.242853:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767007.242858:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4384
00000100:00080000:3.0:1775767007.242863:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767007.242867:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767007.242878:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767007.242894:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744e00 x1862022114617216/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775767007.242905:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114617088:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775767007.242912:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767007.242915:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775767007.242917:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767007.242922:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4384
00000100:00080000:3.0:1775767007.242926:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767007.242930:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767007.242934:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767007.242944:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744380 x1862022114617344/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767007.242953:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767007.242957:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775767007.242958:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767007.242962:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4384
00000100:00080000:3.0:1775767007.242966:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767007.242969:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000800:00000100:0.0:1775767007.242974:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000008:00100000:3.0:1775767007.242974:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775767007.242983:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746a00 x1862022114617472/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767007.242992:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767007.242996:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775767007.242997:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767007.243001:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4384
00000100:00080000:3.0:1775767007.243004:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767007.243008:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775767007.243021:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767007/real 1775767007]  req@ffff9f1485747b80 x1862022114617088/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767032 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:3.0:1775767007.243024:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775767007.243036:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746300 x1862022114617600/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775767007.243042:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747b80 x1862022114617088/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767032 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767007.243046:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767007.243050:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775767007.243053:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767007.243055:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767007.243056:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4384
00000100:00080000:0.0:1775767007.243059:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:3.0:1775767007.243060:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1775767007.243064:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114617088:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775767007.243064:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767007.243069:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767007.243079:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745880 x1862022114617728/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767007.243088:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1775767007.243089:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114617216:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775767007.243092:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775767007.243094:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767007.243098:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4384
00000100:00080000:3.0:1775767007.243101:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767007.243105:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767007.243110:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000800:00000100:0.0:1775767007.243113:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767007.243121:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747480 x1862022114617856/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775767007.243126:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767007/real 1775767007]  req@ffff9f1485744e00 x1862022114617216/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767032 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767007.243139:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1775767007.243140:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744e00 x1862022114617216/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767032 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767007.243143:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775767007.243145:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767007.243149:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4384
00000100:00080000:0.0:1775767007.243150:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767007.243154:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00080000:3.0:1775767007.243154:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767007.243158:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775767007.243159:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114617216:192.168.202.110@tcp:38:kworker.0
00000008:00100000:3.0:1775767007.243163:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775767007.243165:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114617344:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767007.243173:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744a80 x1862022114617984/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1775767007.243185:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767007.243196:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767007/real 1775767007]  req@ffff9f1485744380 x1862022114617344/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767032 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767007.243208:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744380 x1862022114617344/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767032 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767007.243218:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767007.243222:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767007.243225:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114617344:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767007.243233:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114617472:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767007.243251:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767007.243260:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767007/real 1775767007]  req@ffff9f1485746a00 x1862022114617472/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767032 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767007.243274:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746a00 x1862022114617472/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767032 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767007.243283:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767007.243286:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767007.243290:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114617472:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775767007.243297:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114617600:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767007.243320:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767007.243333:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767007/real 1775767007]  req@ffff9f1485746300 x1862022114617600/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767032 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767007.243346:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746300 x1862022114617600/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767032 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767007.243352:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767007.243355:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767007.243357:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114617600:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775767007.243371:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114617728:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767007.243388:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767007.243396:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767007/real 1775767007]  req@ffff9f1485745880 x1862022114617728/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767032 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767007.243405:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745880 x1862022114617728/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767032 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767007.243410:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767007.243413:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767007.243415:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114617728:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767007.243419:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114617856:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767007.243429:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767007.243435:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767007/real 1775767007]  req@ffff9f1485747480 x1862022114617856/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767032 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767007.243443:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747480 x1862022114617856/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767032 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767007.243449:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767007.243451:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767007.243453:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114617856:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767007.243457:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114617984:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767007.243466:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767007.243472:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767007/real 1775767007]  req@ffff9f1485744a80 x1862022114617984/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767032 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767007.243479:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744a80 x1862022114617984/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767032 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767007.243484:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767007.243486:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767007.243489:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114617984:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775767007.498710:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767008.266709:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767008.522773:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767009.290696:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775767009.546753:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767010.314732:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767010.570829:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767011.338700:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767011.594807:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1775767012.362654:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1775767012.362660:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775767012.362668:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775767012.362678:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767012.362687:0:8:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4389
00000100:00080000:3.0:1775767012.362698:0:8:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767012.362704:0:8:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775767012.362749:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747100 x1862022114618112/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767012.362773:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767012.362776:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775767012.362777:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767012.362780:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4389
00000100:00080000:3.0:1775767012.362783:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767012.362788:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767012.362797:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767012.362806:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114618112:192.168.202.110@tcp:250:kworker.0
00000100:00100000:3.0:1775767012.362810:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745c00 x1862022114618240/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767012.362818:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767012.362821:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775767012.362822:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767012.362825:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4389
00000100:00080000:3.0:1775767012.362827:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767012.362829:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767012.362832:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767012.362841:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745180 x1862022114618368/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767012.362847:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767012.362849:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775767012.362850:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767012.362852:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4389
00000100:00080000:3.0:1775767012.362854:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767012.362857:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767012.362861:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775767012.362870:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746680 x1862022114618496/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1775767012.362876:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775767012.362882:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767012.362885:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775767012.362887:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767012.362891:0:8:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4389
00000100:00080000:3.0:1775767012.362897:0:8:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767012.362900:0:8:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767012.362906:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775767012.362916:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767012/real 1775767012]  req@ffff9f1485747100 x1862022114618112/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767037 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767012.362921:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744000 x1862022114618624/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775767012.362940:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747100 x1862022114618112/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767037 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767012.362946:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767012.362956:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775767012.362959:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767012.362963:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767012.362965:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4389
00000100:00080000:0.0:1775767012.362970:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:3.0:1775767012.362970:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767012.362978:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775767012.362979:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114618112:192.168.202.110@tcp:250:kworker.0
00000008:00100000:3.0:1775767012.362985:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767012.362997:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747800 x1862022114618752/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775767012.363002:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114618240:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775767012.363011:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767012.363014:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775767012.363016:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767012.363020:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4389
00000100:00080000:3.0:1775767012.363023:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000800:00000100:0.0:1775767012.363027:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775767012.363030:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767012.363036:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767012.363043:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767012/real 1775767012]  req@ffff9f1485745c00 x1862022114618240/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767037 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767012.363048:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745f80 x1862022114618880/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767012.363062:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1775767012.363064:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745c00 x1862022114618240/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767037 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767012.363066:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775767012.363069:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767012.363077:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4389
00000100:00080000:0.0:1775767012.363081:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767012.363081:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767012.363087:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00080000:0.0:1775767012.363090:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000008:00100000:3.0:1775767012.363092:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775767012.363095:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114618240:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767012.363104:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114618368:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767012.363105:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745500 x1862022114619008/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1775767012.363126:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767012.363135:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767012/real 1775767012]  req@ffff9f1485745180 x1862022114618368/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767037 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767012.363145:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745180 x1862022114618368/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767037 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767012.363151:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767012.363153:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767012.363155:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114618368:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767012.363159:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114618496:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767012.363170:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767012.363175:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767012/real 1775767012]  req@ffff9f1485746680 x1862022114618496/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767037 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767012.363182:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746680 x1862022114618496/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767037 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767012.363187:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767012.363189:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767012.363191:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114618496:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775767012.363194:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114618624:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767012.363203:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767012.363208:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767012/real 1775767012]  req@ffff9f1485744000 x1862022114618624/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767037 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767012.363215:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744000 x1862022114618624/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767037 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767012.363219:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767012.363221:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767012.363222:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114618624:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775767012.363233:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114618752:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767012.363242:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767012.363247:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767012/real 1775767012]  req@ffff9f1485747800 x1862022114618752/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767037 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767012.363255:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747800 x1862022114618752/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767037 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767012.363260:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767012.363262:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767012.363264:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114618752:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767012.363267:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114618880:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767012.363276:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767012.363281:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767012/real 1775767012]  req@ffff9f1485745f80 x1862022114618880/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767037 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767012.363288:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745f80 x1862022114618880/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767037 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767012.363292:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767012.363294:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767012.363296:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114618880:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767012.363299:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114619008:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767012.363307:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767012.363311:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767012/real 1775767012]  req@ffff9f1485745500 x1862022114619008/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767037 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767012.363318:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745500 x1862022114619008/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767037 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767012.363327:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767012.363329:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767012.363331:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114619008:192.168.202.110@tcp:8:kworker.0
00010000:00010000:0.0:1775767012.618699:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767013.386689:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767013.642841:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767014.410664:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767014.666780:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767015.434656:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767015.690767:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767016.458625:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775767016.714768:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767017.482728:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:1.0:1775767017.482768:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767017.482778:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1775767017.482785:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767017.482799:0:8:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4394
00000100:00080000:1.0:1775767017.482813:0:8:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767017.482823:0:8:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775767017.482884:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ffb80 x1862022114619136/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767017.482936:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767017.482942:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775767017.482945:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767017.482953:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4394
00000100:00080000:1.0:1775767017.482961:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767017.482967:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767017.482981:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767017.483004:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ffb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114619136:192.168.202.110@tcp:250:kworker.0
00000100:00100000:1.0:1775767017.483005:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fea00 x1862022114619264/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767017.483028:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767017.483034:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775767017.483036:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767017.483043:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4394
00000100:00080000:1.0:1775767017.483049:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767017.483055:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767017.483062:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767017.483089:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff800 x1862022114619392/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1775767017.483098:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:1.0:1775767017.483110:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767017.483114:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775767017.483117:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767017.483123:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4394
00000100:00080000:1.0:1775767017.483129:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767017.483135:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767017.483143:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775767017.483144:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767017/real 1775767017]  req@ffff9f14ab4ffb80 x1862022114619136/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767042 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767017.483165:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ffb80 x1862022114619136/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767042 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767017.483193:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00100000:1.0:1775767017.483194:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdf80 x1862022114619520/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767017.483206:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767017.483217:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ffb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114619136:192.168.202.110@tcp:250:kworker.0
00000100:00080000:1.0:1775767017.483220:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767017.483231:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1775767017.483234:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767017.483242:0:8:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4394
00000100:00080000:1.0:1775767017.483248:0:8:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767017.483254:0:8:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775767017.483259:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114619264:192.168.202.110@tcp:38:kworker.0
00000008:00100000:1.0:1775767017.483265:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775767017.483288:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ffb80 x1862022114619648/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1775767017.483299:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:1.0:1775767017.483313:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1775767017.483319:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767017/real 1775767017]  req@ffff9f14ab4fea00 x1862022114619264/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767042 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767017.483321:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775767017.483326:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767017.483338:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4394
00000100:00080000:1.0:1775767017.483348:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767017.483371:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767017.483385:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767017.483392:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fea00 x1862022114619264/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767042 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767017.483407:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00100000:1.0:1775767017.483408:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fce00 x1862022114619776/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767017.483421:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767017.483433:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114619264:192.168.202.110@tcp:38:kworker.0
00000100:00080000:1.0:1775767017.483435:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767017.483446:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775767017.483448:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775767017.483452:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114619392:192.168.202.110@tcp:38:kworker.0
00000100:00080000:1.0:1775767017.483459:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4394
00000100:00080000:1.0:1775767017.483467:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767017.483473:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767017.483482:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000800:00000100:0.0:1775767017.483495:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767017.483504:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fca80 x1862022114619904/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767017.483519:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1775767017.483521:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767017/real 1775767017]  req@ffff9f14ab4ff800 x1862022114619392/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767042 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767017.483527:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775767017.483533:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767017.483547:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4394
00000100:00080000:1.0:1775767017.483554:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1775767017.483561:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff800 x1862022114619392/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767042 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767017.483562:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00080000:0.0:1775767017.483608:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767017.483615:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767017.483621:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114619392:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767017.483635:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114619520:192.168.202.110@tcp:8:kworker.0
00000008:00100000:1.0:1775767017.483636:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775767017.483664:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fed80 x1862022114620032/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1775767017.483680:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767017.483721:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767017/real 1775767017]  req@ffff9f14ab4fdf80 x1862022114619520/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767042 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767017.483736:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdf80 x1862022114619520/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767042 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767017.483751:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767017.483756:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767017.483760:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114619520:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775767017.483778:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ffb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114619648:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767017.483795:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767017.483803:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767017/real 1775767017]  req@ffff9f14ab4ffb80 x1862022114619648/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767042 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767017.483813:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ffb80 x1862022114619648/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767042 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767017.483820:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767017.483823:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767017.483825:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ffb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114619648:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775767017.483830:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114619776:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767017.483843:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767017.483850:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767017/real 1775767017]  req@ffff9f14ab4fce00 x1862022114619776/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767042 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767017.483859:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fce00 x1862022114619776/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767042 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767017.483866:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767017.483868:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767017.483871:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114619776:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767017.483876:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114619904:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767017.483906:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767017.483913:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767017/real 1775767017]  req@ffff9f14ab4fca80 x1862022114619904/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767042 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767017.483921:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fca80 x1862022114619904/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767042 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767017.483928:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767017.483930:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767017.483932:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114619904:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767017.483936:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114620032:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767017.483947:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767017.483953:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767017/real 1775767017]  req@ffff9f14ab4fed80 x1862022114620032/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767042 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767017.483961:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fed80 x1862022114620032/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767042 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767017.483968:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767017.483970:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767017.483973:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114620032:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775767017.738744:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767018.506719:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767018.762835:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767019.530638:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:0.0:1775767019.786757:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767020.554623:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767020.810777:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767021.578692:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767021.834726:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767022.602703:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775767022.602710:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767022.602716:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775767022.602720:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767022.602729:0:8:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4400
00000100:00080000:3.0:1775767022.602740:0:8:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767022.602746:0:8:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775767022.602781:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744700 x1862022114620160/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767022.602804:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767022.602807:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775767022.602808:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767022.602811:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4400
00000100:00080000:3.0:1775767022.602814:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767022.602816:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767022.602824:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767022.602835:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745500 x1862022114620288/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775767022.602845:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114620160:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775767022.602845:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767022.602852:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775767022.602855:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767022.602859:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4400
00000100:00080000:3.0:1775767022.602861:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767022.602863:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767022.602866:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767022.602873:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745f80 x1862022114620416/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767022.602879:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767022.602881:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775767022.602883:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767022.602885:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4400
00000100:00080000:3.0:1775767022.602888:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767022.602891:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767022.602894:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775767022.602900:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747800 x1862022114620544/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767022.602906:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767022.602909:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775767022.602910:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767022.602912:0:8:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4400
00000100:00080000:3.0:1775767022.602914:0:8:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767022.602916:0:8:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767022.602918:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:0.0:1775767022.602925:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767022.602926:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744000 x1862022114620672/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767022.602934:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767022.602936:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775767022.602937:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767022.602940:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4400
00000100:00080000:3.0:1775767022.602942:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767022.602946:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767022.602949:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767022.602955:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746680 x1862022114620800/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767022.602960:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767022.602962:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775767022.602964:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767022.602967:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4400
00000100:00080000:3.0:1775767022.602971:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767022.602975:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775767022.602977:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767022/real 1775767022]  req@ffff9f1485744700 x1862022114620160/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767047 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:3.0:1775767022.602979:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767022.602990:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745180 x1862022114620928/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767022.602999:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767022.603002:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775767022.603004:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767022.603008:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4400
00000100:00100000:0.0:1775767022.603009:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744700 x1862022114620160/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767047 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767022.603011:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767022.603017:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767022.603023:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00080000:0.0:1775767022.603030:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00100000:3.0:1775767022.603032:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745c00 x1862022114621056/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767022.603036:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767022.603041:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114620160:192.168.202.110@tcp:250:kworker.0
00000100:00100000:0.0:1775767022.603062:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114620288:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767022.603083:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767022.603093:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767022/real 1775767022]  req@ffff9f1485745500 x1862022114620288/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767047 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767022.603101:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745500 x1862022114620288/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767047 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767022.603107:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767022.603109:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767022.603111:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114620288:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767022.603115:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114620416:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767022.603126:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767022.603131:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767022/real 1775767022]  req@ffff9f1485745f80 x1862022114620416/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767047 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767022.603139:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745f80 x1862022114620416/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767047 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767022.603143:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767022.603146:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767022.603148:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114620416:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767022.603151:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114620544:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767022.603161:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767022.603166:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767022/real 1775767022]  req@ffff9f1485747800 x1862022114620544/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767047 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767022.603174:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747800 x1862022114620544/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767047 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767022.603182:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767022.603186:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767022.603189:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114620544:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775767022.603195:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114620672:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767022.603213:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767022.603224:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767022/real 1775767022]  req@ffff9f1485744000 x1862022114620672/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767047 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767022.603244:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744000 x1862022114620672/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767047 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767022.603249:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767022.603251:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767022.603253:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114620672:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775767022.603257:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114620800:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767022.603272:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767022.603279:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767022/real 1775767022]  req@ffff9f1485746680 x1862022114620800/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767047 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767022.603286:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746680 x1862022114620800/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767047 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767022.603291:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767022.603293:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767022.603303:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114620800:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767022.603307:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114620928:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767022.603318:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767022.603324:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767022/real 1775767022]  req@ffff9f1485745180 x1862022114620928/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767047 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767022.603332:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745180 x1862022114620928/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767047 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767022.603341:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767022.603343:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767022.603345:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114620928:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767022.603348:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114621056:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767022.603358:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767022.603362:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767022/real 1775767022]  req@ffff9f1485745c00 x1862022114621056/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767047 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767022.603369:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745c00 x1862022114621056/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767047 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767022.603374:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767022.603376:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767022.603378:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114621056:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775767022.858777:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767023.626693:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775767023.882750:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767024.650653:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767024.906784:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767025.674726:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767025.930760:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767026.698649:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:0.0:1775767026.954710:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1775767027.722795:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1775767027.722796:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775767027.722809:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775767027.722814:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767027.722856:0:8:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4405
00000100:00080000:3.0:1775767027.722878:0:8:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767027.722891:0:8:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775767027.722943:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747100 x1862022114621184/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767027.722965:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767027.722968:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775767027.722970:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767027.722973:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4405
00000100:00080000:3.0:1775767027.722976:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767027.722979:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767027.722996:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767027.723000:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114621184:192.168.202.110@tcp:250:kworker.0
00000100:00100000:3.0:1775767027.723011:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744a80 x1862022114621312/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767027.723028:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767027.723038:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775767027.723039:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767027.723043:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4405
00000100:00080000:3.0:1775767027.723046:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767027.723048:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767027.723052:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767027.723062:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747480 x1862022114621440/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1775767027.723070:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775767027.723074:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767027.723078:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775767027.723079:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767027.723083:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4405
00000100:00080000:3.0:1775767027.723086:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767027.723089:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767027.723093:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775767027.723102:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745880 x1862022114621568/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767027.723109:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767027.723112:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00100000:0.0:1775767027.723115:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767027/real 1775767027]  req@ffff9f1485747100 x1862022114621184/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767052 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767027.723115:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767027.723120:0:8:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4405
00000100:00080000:3.0:1775767027.723125:0:8:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767027.723131:0:8:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767027.723137:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775767027.723151:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746300 x1862022114621696/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775767027.723156:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747100 x1862022114621184/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767052 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767027.723170:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767027.723176:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775767027.723178:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767027.723179:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767027.723183:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4405
00000100:00080000:3.0:1775767027.723187:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767027.723190:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:3.0:1775767027.723194:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767027.723198:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767027.723202:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114621184:192.168.202.110@tcp:250:kworker.0
00000100:00100000:3.0:1775767027.723209:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746a00 x1862022114621824/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767027.723217:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767027.723220:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775767027.723221:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767027.723224:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4405
00000100:00080000:3.0:1775767027.723226:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767027.723228:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767027.723231:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767027.723236:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114621312:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767027.723240:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744380 x1862022114621952/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767027.723251:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767027.723253:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775767027.723255:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767027.723257:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4405
00000100:00080000:3.0:1775767027.723260:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767027.723263:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767027.723266:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:0.0:1775767027.723276:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767027.723278:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744e00 x1862022114622080/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775767027.723293:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767027/real 1775767027]  req@ffff9f1485744a80 x1862022114621312/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767052 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767027.723305:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744a80 x1862022114621312/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767052 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767027.723312:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767027.723314:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767027.723317:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114621312:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767027.723321:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114621440:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767027.723335:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767027.723341:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767027/real 1775767027]  req@ffff9f1485747480 x1862022114621440/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767052 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767027.723348:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747480 x1862022114621440/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767052 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767027.723358:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767027.723361:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767027.723363:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114621440:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767027.723366:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114621568:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767027.723377:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767027.723383:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767027/real 1775767027]  req@ffff9f1485745880 x1862022114621568/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767052 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767027.723390:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745880 x1862022114621568/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767052 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767027.723396:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767027.723398:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767027.723400:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114621568:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775767027.723403:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114621696:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767027.723430:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767027.723437:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767027/real 1775767027]  req@ffff9f1485746300 x1862022114621696/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767052 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767027.723445:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746300 x1862022114621696/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767052 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767027.723451:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767027.723453:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767027.723457:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114621696:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775767027.723461:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114621824:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767027.723473:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767027.723479:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767027/real 1775767027]  req@ffff9f1485746a00 x1862022114621824/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767052 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767027.723487:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746a00 x1862022114621824/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767052 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767027.723494:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767027.723496:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767027.723498:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114621824:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767027.723513:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114621952:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767027.723524:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767027.723531:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767027/real 1775767027]  req@ffff9f1485744380 x1862022114621952/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767052 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767027.723539:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744380 x1862022114621952/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767052 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767027.723546:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767027.723548:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767027.723550:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114621952:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767027.723555:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114622080:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767027.723566:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767027.723629:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767027/real 1775767027]  req@ffff9f1485744e00 x1862022114622080/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767052 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767027.723638:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744e00 x1862022114622080/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767052 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767027.723644:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767027.723646:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767027.723648:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114622080:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775767027.978687:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767028.746696:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767029.003795:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767029.770723:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767030.026757:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767030.794690:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775767031.050750:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767031.818673:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767032.074825:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1775767032.842670:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1775767032.842675:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775767032.842683:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775767032.842688:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767032.842697:0:8:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4410
00000100:00080000:3.0:1775767032.842711:0:8:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767032.842718:0:8:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775767032.842762:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747b80 x1862022114622208/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767032.842790:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767032.842793:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775767032.842795:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767032.842798:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4410
00000100:00080000:3.0:1775767032.842801:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767032.842804:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767032.842814:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767032.842823:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114622208:192.168.202.110@tcp:250:kworker.0
00000100:00100000:3.0:1775767032.842828:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744e00 x1862022114622336/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767032.842836:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767032.842839:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775767032.842840:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767032.842844:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4410
00000100:00080000:3.0:1775767032.842846:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767032.842848:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767032.842852:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767032.842861:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744380 x1862022114622464/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767032.842867:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767032.842869:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775767032.842870:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767032.842872:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4410
00000100:00080000:3.0:1775767032.842875:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767032.842879:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767032.842883:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775767032.842890:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746a00 x1862022114622592/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767032.842897:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000800:00000100:0.0:1775767032.842898:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775767032.842901:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775767032.842905:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767032.842910:0:8:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4410
00000100:00080000:3.0:1775767032.842912:0:8:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767032.842915:0:8:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767032.842918:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775767032.842926:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746300 x1862022114622720/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767032.842931:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767032.842934:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775767032.842934:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767032.842937:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4410
00000100:00080000:3.0:1775767032.842939:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767032.842943:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767032.842947:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767032.842951:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767032/real 1775767032]  req@ffff9f1485747b80 x1862022114622208/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767057 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767032.842960:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745880 x1862022114622848/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775767032.842971:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747b80 x1862022114622208/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767057 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767032.842978:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767032.842984:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775767032.842986:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767032.842987:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767032.842991:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4410
00000100:00080000:3.0:1775767032.842994:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767032.842995:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:3.0:1775767032.842999:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775767032.843012:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114622208:192.168.202.110@tcp:250:kworker.0
00000008:00100000:3.0:1775767032.843019:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767032.843033:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747480 x1862022114622976/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775767032.843035:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114622336:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775767032.843050:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767032.843053:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775767032.843055:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767032.843059:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4410
00000800:00000100:0.0:1775767032.843063:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775767032.843063:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767032.843070:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767032.843074:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775767032.843079:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767032/real 1775767032]  req@ffff9f1485744e00 x1862022114622336/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767057 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767032.843085:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744a80 x1862022114623104/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775767032.843104:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744e00 x1862022114622336/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767057 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767032.843111:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767032.843113:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767032.843117:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114622336:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767032.843122:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114622464:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767032.843134:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767032.843140:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767032/real 1775767032]  req@ffff9f1485744380 x1862022114622464/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767057 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767032.843147:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744380 x1862022114622464/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767057 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767032.843152:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767032.843154:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767032.843156:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114622464:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767032.843159:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114622592:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767032.843170:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767032.843176:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767032/real 1775767032]  req@ffff9f1485746a00 x1862022114622592/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767057 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767032.843183:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746a00 x1862022114622592/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767057 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767032.843189:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767032.843192:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767032.843194:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114622592:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775767032.843197:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114622720:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767032.843213:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767032.843219:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767032/real 1775767032]  req@ffff9f1485746300 x1862022114622720/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767057 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767032.843225:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746300 x1862022114622720/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767057 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767032.843230:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767032.843232:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767032.843234:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114622720:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775767032.843238:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114622848:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767032.843247:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767032.843252:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767032/real 1775767032]  req@ffff9f1485745880 x1862022114622848/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767057 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767032.843258:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745880 x1862022114622848/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767057 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767032.843264:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767032.843265:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767032.843268:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114622848:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767032.843279:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114622976:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767032.843290:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767032.843295:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767032/real 1775767032]  req@ffff9f1485747480 x1862022114622976/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767057 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767032.843303:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747480 x1862022114622976/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767057 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767032.843308:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767032.843309:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767032.843311:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114622976:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767032.843314:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114623104:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767032.843324:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767032.843329:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767032/real 1775767032]  req@ffff9f1485744a80 x1862022114623104/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767057 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767032.843337:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744a80 x1862022114623104/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767057 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767032.843342:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767032.843344:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767032.843345:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114623104:192.168.202.110@tcp:8:kworker.0
00010000:00010000:3.0:1775767033.098807:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767033.866701:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:0.0:1775767034.122776:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767034.890723:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767035.146743:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767035.914638:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:0.0:1775767036.170731:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767036.938636:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767037.194713:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767037.962641:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:1.0:1775767037.962649:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767037.962655:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1775767037.962659:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767037.962667:0:8:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4415
00000100:00080000:1.0:1775767037.962678:0:8:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767037.962684:0:8:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775767037.962720:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdc00 x1862022114623232/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767037.962746:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767037.962749:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775767037.962750:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767037.962753:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4415
00000100:00080000:1.0:1775767037.962755:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767037.962758:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767037.962766:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767037.962778:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114623232:192.168.202.110@tcp:250:kworker.0
00000100:00100000:1.0:1775767037.962778:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fc380 x1862022114623360/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767037.962791:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767037.962794:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775767037.962795:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767037.962797:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4415
00000100:00080000:1.0:1775767037.962801:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767037.962803:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767037.962806:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767037.962815:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fd880 x1862022114623488/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767037.962821:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767037.962823:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775767037.962824:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767037.962827:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4415
00000100:00080000:1.0:1775767037.962837:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767037.962840:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767037.962844:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:0.0:1775767037.962848:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767037.962853:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fe300 x1862022114623616/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767037.962861:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767037.962864:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1775767037.962866:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767037.962868:0:8:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4415
00000100:00080000:1.0:1775767037.962871:0:8:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767037.962873:0:8:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767037.962876:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775767037.962886:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767037/real 1775767037]  req@ffff9f14ab4fdc00 x1862022114623232/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767062 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767037.962887:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff100 x1862022114623744/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767037.962919:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767037.962923:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775767037.962924:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767037.962929:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4415
00000100:00100000:0.0:1775767037.962933:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdc00 x1862022114623232/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767062 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767037.962934:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767037.962946:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767037.962949:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767037.962952:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114623232:192.168.202.110@tcp:250:kworker.0
00000100:00100000:0.0:1775767037.962970:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114623360:192.168.202.110@tcp:38:kworker.0
00000100:00080000:1.0:1775767037.962974:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767037.962980:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767037.962994:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdc00 x1862022114623872/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1775767037.962997:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767037.963010:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767037/real 1775767037]  req@ffff9f14ab4fc380 x1862022114623360/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767062 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767037.963015:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767037.963041:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775767037.963044:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775767037.963051:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fc380 x1862022114623360/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767062 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767037.963051:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4415
00000100:00080000:1.0:1775767037.963057:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767037.963065:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00080000:0.0:1775767037.963067:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000008:00100000:1.0:1775767037.963070:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00080000:0.0:1775767037.963074:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767037.963079:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114623360:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775767037.963086:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fed80 x1862022114624000/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775767037.963092:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114623488:192.168.202.110@tcp:38:kworker.0
00000100:00080000:1.0:1775767037.963105:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767037.963108:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775767037.963109:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767037.963114:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4415
00000100:00080000:1.0:1775767037.963118:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000800:00000100:0.0:1775767037.963122:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:1.0:1775767037.963126:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767037.963133:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775767037.963138:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767037/real 1775767037]  req@ffff9f14ab4fd880 x1862022114623488/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767062 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767037.963145:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fca80 x1862022114624128/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775767037.963180:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fd880 x1862022114623488/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767062 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767037.963190:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767037.963195:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767037.963199:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114623488:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767037.963205:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114623616:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767037.963227:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767037.963237:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767037/real 1775767037]  req@ffff9f14ab4fe300 x1862022114623616/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767062 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767037.963245:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fe300 x1862022114623616/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767062 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767037.963250:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767037.963252:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767037.963255:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114623616:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775767037.963259:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114623744:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767037.963271:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767037.963276:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767037/real 1775767037]  req@ffff9f14ab4ff100 x1862022114623744/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767062 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767037.963284:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff100 x1862022114623744/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767062 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767037.963289:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767037.963291:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767037.963293:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114623744:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775767037.963305:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114623872:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767037.963315:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767037.963320:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767037/real 1775767037]  req@ffff9f14ab4fdc00 x1862022114623872/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767062 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767037.963328:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdc00 x1862022114623872/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767062 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767037.963333:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767037.963335:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767037.963337:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114623872:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767037.963340:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114624000:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767037.963350:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767037.963355:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767037/real 1775767037]  req@ffff9f14ab4fed80 x1862022114624000/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767062 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767037.963362:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fed80 x1862022114624000/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767062 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767037.963372:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767037.963374:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767037.963376:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114624000:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767037.963379:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114624128:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767037.963387:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767037.963392:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767037/real 1775767037]  req@ffff9f14ab4fca80 x1862022114624128/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767062 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767037.963398:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fca80 x1862022114624128/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767062 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767037.963403:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767037.963405:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767037.963407:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114624128:192.168.202.110@tcp:8:kworker.0
00010000:00010000:1.0:1775767038.218705:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767038.986683:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767039.242761:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767040.010647:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767040.266779:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767041.034641:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:0.0:1775767041.290687:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767042.058665:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767042.314668:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767043.082616:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775767043.082693:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767043.082702:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775767043.082707:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767043.082717:0:8:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4420
00000100:00080000:3.0:1775767043.082729:0:8:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767043.082735:0:8:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775767043.082779:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747100 x1862022114624256/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767043.082809:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767043.082813:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775767043.082815:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767043.082820:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4420
00000100:00080000:3.0:1775767043.082823:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767043.082828:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767043.082840:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767043.082853:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745c00 x1862022114624384/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767043.082863:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767043.082867:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775767043.082869:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775767043.082873:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114624256:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775767043.082873:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4420
00000100:00080000:3.0:1775767043.082877:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767043.082880:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767043.082885:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767043.082894:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745180 x1862022114624512/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767043.082908:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767043.082911:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775767043.082913:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767043.082916:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4420
00000100:00080000:3.0:1775767043.082919:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767043.082923:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767043.082927:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775767043.082936:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746680 x1862022114624640/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767043.082944:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000800:00000100:0.0:1775767043.082947:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775767043.082947:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775767043.082950:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767043.082952:0:8:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4420
00000100:00080000:3.0:1775767043.082955:0:8:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767043.082958:0:8:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767043.082963:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775767043.082971:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744000 x1862022114624768/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767043.082980:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767043.082983:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775767043.082985:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767043.082989:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4420
00000100:00080000:3.0:1775767043.082992:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1775767043.082997:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767043/real 1775767043]  req@ffff9f1485747100 x1862022114624256/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767068 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767043.082998:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767043.083003:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767043.083014:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747800 x1862022114624896/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775767043.083017:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747100 x1862022114624256/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767068 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767043.083024:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767043.083028:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775767043.083030:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767043.083034:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4420
00000100:00080000:0.0:1775767043.083036:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767043.083037:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767043.083040:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:3.0:1775767043.083040:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775767043.083045:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114624256:192.168.202.110@tcp:250:kworker.0
00000008:00100000:3.0:1775767043.083045:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767043.083054:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745f80 x1862022114625024/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767043.083062:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767043.083065:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775767043.083066:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767043.083070:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4420
00000100:00100000:0.0:1775767043.083071:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114624384:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775767043.083074:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767043.083077:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767043.083082:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775767043.083092:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745500 x1862022114625152/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1775767043.083093:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767043.083105:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767043/real 1775767043]  req@ffff9f1485745c00 x1862022114624384/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767068 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767043.083117:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745c00 x1862022114624384/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767068 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767043.083125:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767043.083128:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767043.083131:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114624384:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767043.083137:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114624512:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767043.083158:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767043.083167:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767043/real 1775767043]  req@ffff9f1485745180 x1862022114624512/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767068 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767043.083175:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745180 x1862022114624512/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767068 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767043.083181:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767043.083183:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767043.083185:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114624512:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767043.083190:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114624640:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767043.083203:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767043.083209:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767043/real 1775767043]  req@ffff9f1485746680 x1862022114624640/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767068 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767043.083217:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746680 x1862022114624640/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767068 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767043.083222:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767043.083224:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767043.083225:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114624640:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775767043.083229:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114624768:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767043.083238:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767043.083243:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767043/real 1775767043]  req@ffff9f1485744000 x1862022114624768/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767068 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767043.083250:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744000 x1862022114624768/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767068 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767043.083255:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767043.083257:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767043.083258:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114624768:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775767043.083261:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114624896:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767043.083271:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767043.083276:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767043/real 1775767043]  req@ffff9f1485747800 x1862022114624896/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767068 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767043.083284:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747800 x1862022114624896/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767068 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767043.083289:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767043.083291:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767043.083293:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114624896:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767043.083296:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114625024:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767043.083315:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767043.083321:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767043/real 1775767043]  req@ffff9f1485745f80 x1862022114625024/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767068 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767043.083328:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745f80 x1862022114625024/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767068 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767043.083333:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767043.083334:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767043.083336:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114625024:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767043.083350:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114625152:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767043.083361:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767043.083367:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767043/real 1775767043]  req@ffff9f1485745500 x1862022114625152/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767068 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767043.083374:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745500 x1862022114625152/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767068 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767043.083384:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767043.083386:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767043.083388:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114625152:192.168.202.110@tcp:8:kworker.0
00010000:00010000:0.0:1775767043.338705:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767044.106665:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767044.362748:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767045.130621:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775767045.386873:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767046.154706:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767046.410769:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767047.178655:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767047.434779:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1775767048.202643:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767048.202650:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1775767048.202654:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00100000:3.0:1775767048.202655:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:2.0:1775767048.202665:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4425
00000100:00080000:2.0:1775767048.202675:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767048.202682:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775767048.202716:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2d80 x1862022114625280/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767048.202741:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767048.202745:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775767048.202747:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767048.202752:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4425
00000100:00080000:2.0:1775767048.202755:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767048.202760:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767048.202769:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775767048.202783:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0380 x1862022114625408/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767048.202795:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767048.202798:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775767048.202800:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767048.202804:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4425
00000100:00080000:2.0:1775767048.202807:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767048.202810:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767048.202815:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767048.202819:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114625280:192.168.202.110@tcp:250:kworker.0
00000100:00100000:2.0:1775767048.202826:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1500 x1862022114625536/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767048.202834:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767048.202836:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775767048.202837:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767048.202840:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4425
00000100:00080000:2.0:1775767048.202842:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767048.202845:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767048.202847:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775767048.202857:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1f80 x1862022114625664/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767048.202863:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767048.202865:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1775767048.202866:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767048.202868:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4425
00000100:00080000:2.0:1775767048.202871:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767048.202874:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767048.202877:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:0.0:1775767048.202888:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767048.202888:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0a80 x1862022114625792/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767048.202896:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767048.202899:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775767048.202900:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767048.202904:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4425
00000100:00080000:2.0:1775767048.202907:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767048.202910:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767048.202913:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775767048.202923:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2680 x1862022114625920/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775767048.202930:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767048/real 1775767048]  req@ffff9f14852d2d80 x1862022114625280/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767073 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767048.202935:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767048.202946:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775767048.202948:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767048.202955:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4425
00000100:00100000:0.0:1775767048.202956:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2d80 x1862022114625280/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767073 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767048.202962:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767048.202968:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767048.202970:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00080000:0.0:1775767048.202975:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767048.202979:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114625280:192.168.202.110@tcp:250:kworker.0
00000008:00100000:2.0:1775767048.202979:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767048.202989:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114625408:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767048.202996:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1180 x1862022114626048/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1775767048.203008:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:2.0:1775767048.203010:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767048.203013:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775767048.203015:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767048.203020:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4425
00000100:00100000:0.0:1775767048.203022:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767048/real 1775767048]  req@ffff9f14852d0380 x1862022114625408/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767073 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767048.203025:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767048.203033:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767048.203037:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775767048.203038:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0380 x1862022114625408/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767073 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767048.203046:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767048.203052:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767048.203053:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2a00 x1862022114626176/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775767048.203057:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114625408:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767048.203084:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114625536:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767048.203097:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767048.203103:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767048/real 1775767048]  req@ffff9f14852d1500 x1862022114625536/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767073 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767048.203110:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1500 x1862022114625536/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767073 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767048.203115:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767048.203117:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767048.203119:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114625536:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767048.203123:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114625664:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767048.203133:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767048.203138:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767048/real 1775767048]  req@ffff9f14852d1f80 x1862022114625664/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767073 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767048.203145:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1f80 x1862022114625664/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767073 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767048.203149:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767048.203151:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767048.203153:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114625664:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775767048.203156:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114625792:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767048.203166:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767048.203171:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767048/real 1775767048]  req@ffff9f14852d0a80 x1862022114625792/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767073 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767048.203178:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0a80 x1862022114625792/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767073 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767048.203183:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767048.203185:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767048.203187:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114625792:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775767048.203192:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114625920:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767048.203209:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767048.203220:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767048/real 1775767048]  req@ffff9f14852d2680 x1862022114625920/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767073 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767048.203232:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2680 x1862022114625920/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767073 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767048.203241:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767048.203244:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767048.203247:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114625920:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767048.203253:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114626048:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767048.203268:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767048.203276:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767048/real 1775767048]  req@ffff9f14852d1180 x1862022114626048/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767073 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767048.203293:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1180 x1862022114626048/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767073 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767048.203299:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767048.203301:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767048.203303:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114626048:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767048.203306:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114626176:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767048.203318:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767048.203323:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767048/real 1775767048]  req@ffff9f14852d2a00 x1862022114626176/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767073 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767048.203330:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2a00 x1862022114626176/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767073 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767048.203336:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767048.203337:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767048.203339:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114626176:192.168.202.110@tcp:8:kworker.0
00010000:00010000:0.0:1775767048.458691:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775767049.226682:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767049.482786:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775767050.250685:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:0.0:1775767050.506760:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775767051.274656:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767051.530835:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775767052.298654:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775767052.554892:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1775767053.322635:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767053.322644:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1775767053.322648:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767053.322656:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4430
00000100:00080000:2.0:1775767053.322667:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1775767053.322720:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:2.0:1775767053.322721:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775767053.322762:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0700 x1862022114626304/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767053.322790:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767053.322795:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775767053.322798:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767053.322804:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4430
00000100:00080000:2.0:1775767053.322810:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767053.322814:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775767053.322820:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114626304:192.168.202.110@tcp:250:kworker.0
00000008:00100000:2.0:1775767053.322824:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775767053.322838:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3800 x1862022114626432/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767053.322848:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767053.322852:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775767053.322855:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767053.322860:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4430
00000100:00080000:2.0:1775767053.322867:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767053.322871:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767053.322874:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000800:00000100:0.0:1775767053.322883:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767053.322884:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2a00 x1862022114626560/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767053.322892:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767053.322895:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775767053.322896:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767053.322900:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4430
00000100:00080000:2.0:1775767053.322906:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767053.322909:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767053.322917:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775767053.322932:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767053/real 1775767053]  req@ffff9f14852d0700 x1862022114626304/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767078 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767053.322932:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1180 x1862022114626688/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767053.322965:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767053.322968:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00100000:0.0:1775767053.322971:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0700 x1862022114626304/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767078 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767053.322971:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767053.322984:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4430
00000100:00080000:2.0:1775767053.322989:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767053.322996:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767053.323004:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00080000:0.0:1775767053.323007:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767053.323013:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767053.323015:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2680 x1862022114626816/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775767053.323020:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114626304:192.168.202.110@tcp:250:kworker.0
00000100:00080000:2.0:1775767053.323031:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767053.323034:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775767053.323035:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767053.323039:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4430
00000100:00080000:2.0:1775767053.323041:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767053.323044:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767053.323047:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767053.323048:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114626432:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767053.323059:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0a80 x1862022114626944/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767053.323071:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000800:00000100:0.0:1775767053.323072:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:2.0:1775767053.323078:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775767053.323080:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767053.323085:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4430
00000100:00080000:2.0:1775767053.323090:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767053.323094:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775767053.323096:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767053/real 1775767053]  req@ffff9f14852d3800 x1862022114626432/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767078 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:2.0:1775767053.323103:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767053.323110:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3800 x1862022114626432/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767078 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767053.323118:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00100000:2.0:1775767053.323119:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1f80 x1862022114627072/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767053.323125:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767053.323135:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114626432:192.168.202.110@tcp:38:kworker.0
00000100:00080000:2.0:1775767053.323139:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1775767053.323142:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114626560:192.168.202.110@tcp:38:kworker.0
00000100:00080000:2.0:1775767053.323146:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775767053.323149:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767053.323156:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4430
00000100:00080000:2.0:1775767053.323161:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000800:00000100:0.0:1775767053.323164:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:2.0:1775767053.323167:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767053.323174:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775767053.323177:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767053/real 1775767053]  req@ffff9f14852d2a00 x1862022114626560/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767078 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767053.323187:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1500 x1862022114627200/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775767053.323192:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2a00 x1862022114626560/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767078 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767053.323205:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767053.323218:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767053.323222:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114626560:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767053.323230:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114626688:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767053.323258:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767053.323272:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767053/real 1775767053]  req@ffff9f14852d1180 x1862022114626688/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767078 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767053.323282:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1180 x1862022114626688/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767078 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767053.323290:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767053.323301:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767053.323303:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114626688:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775767053.323322:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114626816:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767053.323338:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767053.323346:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767053/real 1775767053]  req@ffff9f14852d2680 x1862022114626816/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767078 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767053.323359:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2680 x1862022114626816/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767078 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767053.323367:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767053.323370:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767053.323374:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114626816:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775767053.323412:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114626944:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767053.323425:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767053.323432:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767053/real 1775767053]  req@ffff9f14852d0a80 x1862022114626944/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767078 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767053.323441:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0a80 x1862022114626944/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767078 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767053.323449:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767053.323452:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767053.323456:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114626944:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767053.323461:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114627072:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767053.323481:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767053.323492:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767053/real 1775767053]  req@ffff9f14852d1f80 x1862022114627072/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767078 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767053.323501:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1f80 x1862022114627072/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767078 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767053.323507:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767053.323509:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767053.323512:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114627072:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767053.323518:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114627200:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767053.323537:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767053.323548:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767053/real 1775767053]  req@ffff9f14852d1500 x1862022114627200/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767078 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767053.323601:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1500 x1862022114627200/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767078 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767053.323618:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767053.323622:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767053.323625:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114627200:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775767053.579045:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775767054.346693:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767054.603675:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775767055.370684:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:0.0:1775767055.626717:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775767056.394645:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767056.650975:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775767057.418616:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:0.0:1775767057.674682:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1775767058.442656:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:2.0:1775767058.442663:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775767058.442665:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775767058.442671:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767058.442681:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4435
00000100:00080000:3.0:1775767058.442691:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767058.442698:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775767058.442736:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744700 x1862022114627328/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767058.442759:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767058.442761:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775767058.442763:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767058.442766:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4435
00000100:00080000:3.0:1775767058.442769:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767058.442772:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767058.442780:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767058.442790:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745500 x1862022114627456/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767058.442798:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767058.442801:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775767058.442804:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775767058.442806:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114627328:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775767058.442808:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4435
00000100:00080000:3.0:1775767058.442813:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767058.442818:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767058.442821:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767058.442828:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745f80 x1862022114627584/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767058.442835:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767058.442838:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775767058.442839:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767058.442842:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4435
00000100:00080000:3.0:1775767058.442844:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767058.442846:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767058.442849:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775767058.442855:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747800 x1862022114627712/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767058.442862:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767058.442864:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775767058.442865:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767058.442868:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4435
00000100:00080000:3.0:1775767058.442871:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000800:00000100:0.0:1775767058.442875:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775767058.442877:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767058.442882:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775767058.442889:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744000 x1862022114627840/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767058.442896:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767058.442898:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775767058.442899:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767058.442902:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4435
00000100:00080000:3.0:1775767058.442904:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767058.442907:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767058.442910:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767058.442917:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767058/real 1775767058]  req@ffff9f1485744700 x1862022114627328/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767083 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767058.442928:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746680 x1862022114627968/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767058.442937:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1775767058.442941:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744700 x1862022114627328/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767083 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767058.442941:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775767058.442943:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767058.442952:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4435
00000100:00080000:3.0:1775767058.442958:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767058.442963:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00080000:0.0:1775767058.442965:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000008:00100000:3.0:1775767058.442966:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00080000:0.0:1775767058.442973:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767058.442978:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745180 x1862022114628096/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775767058.442980:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114627328:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775767058.442994:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767058.442996:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775767058.442998:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767058.443001:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4435
00000100:00080000:3.0:1775767058.443003:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767058.443005:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767058.443009:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775767058.443010:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114627456:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767058.443019:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745c00 x1862022114628224/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1775767058.443037:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767058.443052:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767058/real 1775767058]  req@ffff9f1485745500 x1862022114627456/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767083 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767058.443066:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745500 x1862022114627456/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767083 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767058.443074:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767058.443078:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767058.443082:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114627456:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767058.443087:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114627584:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767058.443103:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767058.443112:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767058/real 1775767058]  req@ffff9f1485745f80 x1862022114627584/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767083 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767058.443124:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745f80 x1862022114627584/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767083 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767058.443133:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767058.443136:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767058.443139:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114627584:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767058.443145:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114627712:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767058.443160:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767058.443169:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767058/real 1775767058]  req@ffff9f1485747800 x1862022114627712/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767083 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767058.443180:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747800 x1862022114627712/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767083 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767058.443188:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767058.443192:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767058.443195:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114627712:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775767058.443201:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114627840:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767058.443216:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767058.443224:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767058/real 1775767058]  req@ffff9f1485744000 x1862022114627840/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767083 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767058.443236:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744000 x1862022114627840/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767083 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767058.443244:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767058.443247:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767058.443250:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114627840:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775767058.443286:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114627968:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767058.443305:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767058.443315:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767058/real 1775767058]  req@ffff9f1485746680 x1862022114627968/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767083 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767058.443326:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746680 x1862022114627968/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767083 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767058.443336:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767058.443338:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767058.443342:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114627968:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767058.443348:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114628096:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767058.443363:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767058.443371:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767058/real 1775767058]  req@ffff9f1485745180 x1862022114628096/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767083 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767058.443390:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745180 x1862022114628096/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767083 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767058.443399:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767058.443402:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767058.443405:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114628096:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767058.443423:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114628224:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767058.443439:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767058.443449:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767058/real 1775767058]  req@ffff9f1485745c00 x1862022114628224/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767083 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767058.443462:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745c00 x1862022114628224/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767083 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767058.443470:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767058.443474:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767058.443477:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114628224:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775767058.698944:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767059.466626:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775767059.722686:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767060.490685:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767060.746719:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767061.514680:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767061.770729:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767062.538665:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:0.0:1775767062.794680:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1775767063.562636:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767063.562644:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00100000:0.0:1775767063.562645:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775767063.562647:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767063.562659:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4441
00000100:00080000:3.0:1775767063.562671:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767063.562678:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775767063.562714:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747100 x1862022114628352/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767063.562736:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767063.562739:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775767063.562741:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767063.562744:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4441
00000100:00080000:3.0:1775767063.562747:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767063.562752:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767063.562760:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767063.562767:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114628352:192.168.202.110@tcp:250:kworker.0
00000100:00100000:3.0:1775767063.562772:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744a80 x1862022114628480/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767063.562780:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767063.562783:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775767063.562784:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767063.562787:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4441
00000100:00080000:3.0:1775767063.562789:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767063.562791:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767063.562794:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767063.562802:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747480 x1862022114628608/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767063.562807:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767063.562809:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775767063.562811:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767063.562813:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4441
00000100:00080000:3.0:1775767063.562816:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767063.562819:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767063.562822:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775767063.562828:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745880 x1862022114628736/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767063.562834:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767063.562837:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775767063.562838:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767063.562840:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4441
00000100:00080000:3.0:1775767063.562842:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767063.562844:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767063.562847:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775767063.562856:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746300 x1862022114628864/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767063.562862:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767063.562864:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775767063.562866:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775767063.562870:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114628480:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775767063.562872:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4441
00000100:00080000:3.0:1775767063.562877:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1775767063.562914:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114628608:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775767063.562924:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767063.562931:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767063.562935:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114628736:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775767063.562949:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746a00 x1862022114628992/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775767063.562957:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114628864:192.168.202.110@tcp:8:kworker.0
00000100:00080000:3.0:1775767063.562960:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767063.562963:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775767063.562964:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767063.562969:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4441
00000100:00080000:3.0:1775767063.562973:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767063.562979:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775767063.562980:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114628992:192.168.202.110@tcp:38:kworker.0
00000008:00100000:3.0:1775767063.562988:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767063.563000:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744380 x1862022114629120/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767063.563008:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767063.563011:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775767063.563012:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775767063.563015:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114629120:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775767063.563018:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4441
00000100:00080000:3.0:1775767063.563023:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767063.563025:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767063.563030:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775767063.563039:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744e00 x1862022114629248/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775767063.563066:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114629248:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775767063.818701:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767064.586619:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:0.0:1775767064.842680:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767065.610681:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767065.866730:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767066.634634:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000400:00000100:2.0:1775767066.634802:0:2392:0:(lib-socket.c:454:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.202.110:988
00000400:00000100:2.0:1775767066.634844:0:2392:0:(acceptor.c:111:lnet_connect_console_error()) Connection to 192.168.202.110@tcp at host 192.168.202.110 was unreachable: the network or that node may be down, or Lustre may be misconfigured.
00000800:00000100:2.0:1775767066.634862:0:2392:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:2.0:1775767066.634946:0:2392:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000100:00100000:0.0:1775767066.634951:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767063/real 1775767066]  req@ffff9f1485747100 x1862022114628352/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767088 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000800:00000100:2.0:1775767066.634962:0:2392:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:2.0:1775767066.634974:0:2392:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000100:00100000:0.0:1775767066.634988:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747100 x1862022114628352/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767088 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000800:00000100:2.0:1775767066.634989:0:2392:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:2.0:1775767066.635001:0:2392:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000100:00080000:0.0:1775767066.635006:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000800:00000100:2.0:1775767066.635012:0:2392:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000100:00080000:0.0:1775767066.635015:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767066.635022:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114628352:192.168.202.110@tcp:250:kworker.0
00000800:00000100:2.0:1775767066.635028:0:2392:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000100:00100000:0.0:1775767066.635092:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767063/real 1775767066]  req@ffff9f1485744e00 x1862022114629248/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767088 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767066.635108:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744e00 x1862022114629248/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767088 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767066.635124:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767066.635128:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767066.635132:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114629248:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775767066.635140:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767063/real 1775767066]  req@ffff9f1485744380 x1862022114629120/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767088 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767066.635154:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744380 x1862022114629120/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767088 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767066.635163:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767066.635196:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767066.635200:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114629120:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767066.635208:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767063/real 1775767066]  req@ffff9f1485746a00 x1862022114628992/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767088 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767066.635222:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746a00 x1862022114628992/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767088 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767066.635245:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767066.635249:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767066.635253:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114628992:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767066.635259:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767063/real 1775767066]  req@ffff9f1485744a80 x1862022114628480/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767088 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767066.635273:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744a80 x1862022114628480/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767088 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767066.635280:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767066.635284:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767066.635287:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114628480:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767066.635294:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767063/real 1775767066]  req@ffff9f1485747480 x1862022114628608/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767088 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767066.635306:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747480 x1862022114628608/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767088 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767066.635314:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767066.635318:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767066.635321:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114628608:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767066.635328:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767063/real 1775767066]  req@ffff9f1485745880 x1862022114628736/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767088 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767066.635341:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745880 x1862022114628736/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767088 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767066.635349:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767066.635352:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767066.635356:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114628736:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775767066.635363:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767063/real 1775767066]  req@ffff9f1485746300 x1862022114628864/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767088 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767066.636356:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746300 x1862022114628864/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767088 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767066.636381:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767066.636386:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767066.636389:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114628864:192.168.202.110@tcp:8:kworker.0
00010000:00010000:1.0:1775767066.890702:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767067.658681:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767067.914674:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1775767068.682615:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767068.682625:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775767068.682629:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767068.682640:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4446
00000100:00100000:2.0:1775767068.682651:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775767068.682651:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767068.682662:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775767068.682693:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747b80 x1862022114629376/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767068.682725:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767068.682729:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775767068.682731:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767068.682736:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4446
00000100:00080000:3.0:1775767068.682742:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767068.682747:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767068.682756:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767068.682773:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746300 x1862022114629504/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775767068.682777:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114629376:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775767068.682794:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767068.682798:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775767068.682799:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767068.682804:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4446
00000100:00080000:3.0:1775767068.682809:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767068.682812:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767068.682818:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767068.682833:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745880 x1862022114629632/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767068.682843:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767068.682846:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775767068.682847:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767068.682851:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4446
00000100:00080000:3.0:1775767068.682855:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000800:00000100:0.0:1775767068.682856:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775767068.682863:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767068.682870:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775767068.682882:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747480 x1862022114629760/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767068.682900:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767068.682904:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00100000:0.0:1775767068.682907:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767068/real 1775767068]  req@ffff9f1485747b80 x1862022114629376/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767093 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767068.682908:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767068.682917:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4446
00000100:00080000:3.0:1775767068.682926:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767068.682933:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767068.682940:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775767068.682957:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744a80 x1862022114629888/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767068.682966:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767068.682970:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775767068.682972:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775767068.682975:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747b80 x1862022114629376/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767093 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767068.682979:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4446
00000100:00080000:3.0:1775767068.682986:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767068.682995:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767068.682995:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767068.683003:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00080000:0.0:1775767068.683004:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767068.683012:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114629376:192.168.202.110@tcp:250:kworker.0
00000100:00100000:3.0:1775767068.683022:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746a00 x1862022114630016/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767068.683033:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767068.683037:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00100000:0.0:1775767068.683041:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114629504:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775767068.683041:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767068.683055:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4446
00000100:00080000:3.0:1775767068.683061:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767068.683065:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767068.683070:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000800:00000100:0.0:1775767068.683077:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767068.683085:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744380 x1862022114630144/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775767068.683096:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767068/real 1775767068]  req@ffff9f1485746300 x1862022114629504/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767093 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767068.683099:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767068.683111:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775767068.683115:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775767068.683120:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746300 x1862022114629504/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767093 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767068.683122:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4446
00000100:00080000:3.0:1775767068.683131:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767068.683139:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00080000:0.0:1775767068.683140:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000008:00100000:3.0:1775767068.683167:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00080000:0.0:1775767068.683168:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767068.683176:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114629504:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767068.683185:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114629632:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767068.683202:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744e00 x1862022114630272/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1775767068.683211:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767068.683232:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767068/real 1775767068]  req@ffff9f1485745880 x1862022114629632/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767093 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767068.683247:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745880 x1862022114629632/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767093 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767068.683256:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767068.683261:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767068.683264:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114629632:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767068.683271:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114629760:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767068.683295:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767068.683308:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767068/real 1775767068]  req@ffff9f1485747480 x1862022114629760/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767093 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767068.683321:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747480 x1862022114629760/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767093 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767068.683330:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767068.683334:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767068.683338:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114629760:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775767068.683345:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114629888:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767068.683367:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767068.683385:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767068/real 1775767068]  req@ffff9f1485744a80 x1862022114629888/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767093 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767068.683399:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744a80 x1862022114629888/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767093 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767068.683407:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767068.683410:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767068.683413:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114629888:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775767068.683419:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114630016:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767068.683441:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767068.683453:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767068/real 1775767068]  req@ffff9f1485746a00 x1862022114630016/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767093 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767068.683465:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746a00 x1862022114630016/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767093 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767068.683474:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767068.683477:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767068.683480:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114630016:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767068.683499:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114630144:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767068.683519:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767068.683530:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767068/real 1775767068]  req@ffff9f1485744380 x1862022114630144/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767093 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767068.683540:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744380 x1862022114630144/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767093 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767068.683553:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767068.683587:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767068.683590:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114630144:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767068.683596:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114630272:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767068.683618:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767068.683630:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767068/real 1775767068]  req@ffff9f1485744e00 x1862022114630272/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767093 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767068.683642:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744e00 x1862022114630272/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767093 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767068.683651:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767068.683655:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767068.683658:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114630272:192.168.202.110@tcp:8:kworker.0
00010000:00010000:3.0:1775767068.938754:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767069.706767:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:0.0:1775767069.962743:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767070.730675:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767070.986669:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767071.754627:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:0.0:1775767072.010707:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767072.778671:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767073.034693:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767073.802643:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775767073.802732:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767073.802747:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775767073.802755:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767073.802769:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4451
00000100:00080000:3.0:1775767073.802786:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767073.802796:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775767073.802858:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747100 x1862022114630400/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767073.802901:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767073.802907:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775767073.802909:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767073.802915:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4451
00000100:00080000:3.0:1775767073.802920:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767073.802926:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767073.802940:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767073.802956:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114630400:192.168.202.110@tcp:250:kworker.0
00000100:00100000:3.0:1775767073.802956:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745c00 x1862022114630528/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767073.803008:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767073.803012:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775767073.803016:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767073.803022:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4451
00000100:00080000:3.0:1775767073.803028:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767073.803032:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767073.803037:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767073.803050:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745180 x1862022114630656/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1775767073.803062:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775767073.803075:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767073.803080:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775767073.803084:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767073.803115:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4451
00000100:00080000:3.0:1775767073.803123:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767073.803128:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767073.803140:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775767073.803149:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767073/real 1775767073]  req@ffff9f1485747100 x1862022114630400/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767098 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767073.803157:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746680 x1862022114630784/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767073.803234:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767073.803239:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775767073.803242:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775767073.803253:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747100 x1862022114630400/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767098 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767073.803253:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4451
00000100:00080000:3.0:1775767073.803263:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767073.803280:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00080000:0.0:1775767073.803284:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000008:00100000:3.0:1775767073.803290:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775767073.803309:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744000 x1862022114630912/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767073.803333:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767073.803350:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114630400:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775767073.803354:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767073.803361:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775767073.803365:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775767073.803378:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114630528:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775767073.803378:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4451
00000100:00080000:3.0:1775767073.803396:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767073.803407:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000800:00000100:0.0:1775767073.803414:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000008:00100000:3.0:1775767073.803417:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767073.803427:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767073/real 1775767073]  req@ffff9f1485745c00 x1862022114630528/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767098 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767073.803450:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745c00 x1862022114630528/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767098 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767073.803452:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747800 x1862022114631040/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767073.803483:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767073.803494:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767073.803504:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114630528:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775767073.803506:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767073.803519:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775767073.803523:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767073.803534:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4451
00000100:00100000:0.0:1775767073.803535:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114630656:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775767073.803547:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000800:00000100:0.0:1775767073.803602:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775767073.803608:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775767073.803618:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767073/real 1775767073]  req@ffff9f1485745180 x1862022114630656/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767098 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:3.0:1775767073.803620:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767073.803643:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745180 x1862022114630656/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767098 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767073.803653:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00100000:3.0:1775767073.803664:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745f80 x1862022114631168/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767073.803665:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767073.803685:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114630656:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775767073.803687:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767073.803697:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775767073.803701:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767073.803708:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4451
00000100:00100000:0.0:1775767073.803713:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114630784:192.168.202.110@tcp:8:kworker.0
00000100:00080000:3.0:1775767073.803717:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767073.803723:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767073.803731:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775767073.803746:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745500 x1862022114631296/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1775767073.803749:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767073.803767:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767073/real 1775767073]  req@ffff9f1485746680 x1862022114630784/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767098 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767073.803783:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746680 x1862022114630784/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767098 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767073.803793:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767073.803797:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767073.803801:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114630784:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775767073.803810:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114630912:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767073.803833:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767073.803842:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767073/real 1775767073]  req@ffff9f1485744000 x1862022114630912/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767098 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767073.803851:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744000 x1862022114630912/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767098 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767073.803856:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767073.803858:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767073.803860:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114630912:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775767073.803875:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114631040:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767073.803887:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767073.803894:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767073/real 1775767073]  req@ffff9f1485747800 x1862022114631040/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767098 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767073.803906:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747800 x1862022114631040/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767098 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767073.803911:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767073.803913:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767073.803914:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114631040:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767073.803918:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114631168:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767073.803928:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767073.803934:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767073/real 1775767073]  req@ffff9f1485745f80 x1862022114631168/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767098 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767073.803942:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745f80 x1862022114631168/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767098 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767073.803946:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767073.803948:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767073.803950:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114631168:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767073.803954:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114631296:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767073.803963:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767073.803974:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767073/real 1775767073]  req@ffff9f1485745500 x1862022114631296/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767098 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767073.803981:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745500 x1862022114631296/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767098 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767073.803990:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767073.803992:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767073.803994:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114631296:192.168.202.110@tcp:8:kworker.0
00010000:00010000:1.0:1775767074.058840:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767074.826717:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767075.082759:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767075.850621:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767076.106774:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767076.874652:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:0.0:1775767077.130672:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767077.898655:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767078.154745:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767078.922636:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:1.0:1775767078.922649:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767078.922657:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1775767078.922662:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767078.922672:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4456
00000100:00080000:1.0:1775767078.922683:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767078.922691:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775767078.922730:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fce00 x1862022114631424/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767078.922762:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767078.922766:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775767078.922768:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767078.922772:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4456
00000100:00080000:1.0:1775767078.922777:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767078.922781:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767078.922790:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767078.922803:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ffb80 x1862022114631552/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775767078.922811:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114631424:192.168.202.110@tcp:250:kworker.0
00000100:00080000:1.0:1775767078.922815:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767078.922818:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775767078.922820:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767078.922823:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4456
00000100:00080000:1.0:1775767078.922826:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767078.922829:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767078.922833:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767078.922843:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdf80 x1862022114631680/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767078.922849:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767078.922852:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775767078.922853:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767078.922855:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4456
00000100:00080000:1.0:1775767078.922859:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767078.922861:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767078.922864:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:0.0:1775767078.922868:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767078.922873:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff800 x1862022114631808/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767078.922880:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767078.922883:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1775767078.922884:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767078.922887:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4456
00000100:00080000:1.0:1775767078.922889:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767078.922891:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767078.922894:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775767078.922907:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767078/real 1775767078]  req@ffff9f14ab4fce00 x1862022114631424/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767103 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767078.922907:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fea00 x1862022114631936/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775767078.922946:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fce00 x1862022114631424/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767103 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767078.922953:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767078.922955:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767078.922961:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775767078.922962:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:1.0:1775767078.922966:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775767078.922968:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114631424:192.168.202.110@tcp:250:kworker.0
00000100:00080000:1.0:1775767078.922975:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4456
00000100:00080000:1.0:1775767078.922981:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767078.922986:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775767078.922990:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ffb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114631552:192.168.202.110@tcp:38:kworker.0
00000008:00100000:1.0:1775767078.922993:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767078.923009:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fce00 x1862022114632064/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1775767078.923021:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:1.0:1775767078.923022:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767078.923027:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775767078.923029:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775767078.923033:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767078/real 1775767078]  req@ffff9f14ab4ffb80 x1862022114631552/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767103 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767078.923035:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4456
00000100:00080000:1.0:1775767078.923086:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767078.923091:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767078.923105:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767078.923121:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fca80 x1862022114632192/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775767078.923125:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ffb80 x1862022114631552/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767103 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767078.923144:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767078.923146:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767078.923148:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775767078.923151:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767078.923152:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00080000:1.0:1775767078.923161:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4456
00000100:00100000:0.0:1775767078.923162:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ffb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114631552:192.168.202.110@tcp:38:kworker.0
00000100:00080000:1.0:1775767078.923166:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1775767078.923171:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114631680:192.168.202.110@tcp:38:kworker.0
00000100:00080000:1.0:1775767078.923172:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767078.923180:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775767078.923194:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fed80 x1862022114632320/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1775767078.923201:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767078.923213:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767078/real 1775767078]  req@ffff9f14ab4fdf80 x1862022114631680/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767103 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767078.923222:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdf80 x1862022114631680/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767103 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767078.923228:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767078.923230:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767078.923232:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114631680:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767078.923236:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114631808:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767078.923246:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767078.923252:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767078/real 1775767078]  req@ffff9f14ab4ff800 x1862022114631808/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767103 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767078.923259:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff800 x1862022114631808/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767103 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767078.923264:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767078.923266:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767078.923267:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114631808:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775767078.923271:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114631936:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767078.923280:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767078.923285:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767078/real 1775767078]  req@ffff9f14ab4fea00 x1862022114631936/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767103 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767078.923293:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fea00 x1862022114631936/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767103 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767078.923298:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767078.923299:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767078.923301:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114631936:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775767078.923312:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114632064:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767078.923321:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767078.923327:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767078/real 1775767078]  req@ffff9f14ab4fce00 x1862022114632064/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767103 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767078.923334:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fce00 x1862022114632064/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767103 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767078.923339:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767078.923340:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767078.923342:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114632064:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767078.923345:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114632192:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767078.923364:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767078.923370:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767078/real 1775767078]  req@ffff9f14ab4fca80 x1862022114632192/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767103 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767078.923377:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fca80 x1862022114632192/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767103 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767078.923383:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767078.923385:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767078.923387:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114632192:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767078.923390:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114632320:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767078.923399:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767078.923404:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767078/real 1775767078]  req@ffff9f14ab4fed80 x1862022114632320/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767103 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767078.923411:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fed80 x1862022114632320/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767103 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767078.923416:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767078.923418:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767078.923419:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114632320:192.168.202.110@tcp:8:kworker.0
00010000:00010000:0.0:1775767079.178706:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767079.946639:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767080.202735:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767080.970628:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775767081.226713:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767081.994643:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767082.250727:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767083.018648:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767083.274722:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767084.042644:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:1.0:1775767084.042647:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767084.042656:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1775767084.042660:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767084.042670:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4461
00000100:00080000:1.0:1775767084.042682:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767084.042690:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775767084.042727:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdc00 x1862022114632448/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767084.042759:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767084.042763:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775767084.042765:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767084.042770:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4461
00000100:00080000:1.0:1775767084.042774:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767084.042780:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767084.042790:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767084.042803:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114632448:192.168.202.110@tcp:250:kworker.0
00000100:00100000:1.0:1775767084.042810:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff100 x1862022114632576/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767084.042824:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767084.042828:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775767084.042831:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767084.042836:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4461
00000100:00080000:1.0:1775767084.042852:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767084.042857:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000800:00000100:0.0:1775767084.042861:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000008:00100000:1.0:1775767084.042866:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767084.042884:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fe300 x1862022114632704/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767084.042897:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1775767084.042900:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767084/real 1775767084]  req@ffff9f14ab4fdc00 x1862022114632448/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767109 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767084.042904:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775767084.042909:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767084.042918:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4461
00000100:00100000:0.0:1775767084.042922:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdc00 x1862022114632448/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767109 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767084.042924:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767084.042934:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00080000:0.0:1775767084.042935:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767084.042942:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000008:00100000:1.0:1775767084.042945:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775767084.042947:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114632448:192.168.202.110@tcp:250:kworker.0
00000100:00100000:1.0:1775767084.042964:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fd880 x1862022114632832/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775767084.042970:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114632576:192.168.202.110@tcp:38:kworker.0
00000100:00080000:1.0:1775767084.042983:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767084.043003:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1775767084.043005:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767084.043012:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4461
00000800:00000100:0.0:1775767084.043017:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:1.0:1775767084.043018:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767084.043025:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775767084.043030:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767084/real 1775767084]  req@ffff9f14ab4ff100 x1862022114632576/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767109 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:1.0:1775767084.043032:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775767084.043045:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff100 x1862022114632576/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767109 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767084.043052:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00100000:1.0:1775767084.043053:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fc380 x1862022114632960/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767084.043059:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767084.043064:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114632576:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767084.043075:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114632704:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767084.043092:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:1.0:1775767084.043092:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767084.043099:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775767084.043101:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775767084.043103:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767084/real 1775767084]  req@ffff9f14ab4fe300 x1862022114632704/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767109 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767084.043114:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4461
00000100:00100000:0.0:1775767084.043119:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fe300 x1862022114632704/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767109 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767084.043122:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767084.043130:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767084.043133:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00080000:0.0:1775767084.043136:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767084.043140:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114632704:192.168.202.110@tcp:38:kworker.0
00000008:00100000:1.0:1775767084.043144:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767084.043152:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114632832:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775767084.043162:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fe300 x1862022114633088/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1775767084.043169:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767084.043178:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767084/real 1775767084]  req@ffff9f14ab4fd880 x1862022114632832/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767109 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767084.043182:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767084.043195:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775767084.043198:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775767084.043199:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fd880 x1862022114632832/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767109 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767084.043208:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4461
00000100:00080000:0.0:1775767084.043210:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767084.043215:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00080000:1.0:1775767084.043216:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1775767084.043219:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114632832:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775767084.043226:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114632960:192.168.202.110@tcp:8:kworker.0
00000100:00080000:1.0:1775767084.043226:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767084.043238:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000800:00000100:0.0:1775767084.043244:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767084.043256:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767084/real 1775767084]  req@ffff9f14ab4fc380 x1862022114632960/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767109 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767084.043256:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff100 x1862022114633216/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775767084.043281:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fc380 x1862022114632960/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767109 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767084.043284:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767084.043294:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775767084.043298:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767084.043301:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767084.043306:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4461
00000100:00080000:0.0:1775767084.043311:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00080000:1.0:1775767084.043313:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1775767084.043317:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114632960:192.168.202.110@tcp:8:kworker.0
00000100:00080000:1.0:1775767084.043322:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767084.043330:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775767084.043338:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114633088:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775767084.043345:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdc00 x1862022114633344/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1775767084.043365:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767084.043381:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767084/real 1775767084]  req@ffff9f14ab4fe300 x1862022114633088/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767109 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767084.043394:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fe300 x1862022114633088/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767109 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767084.043403:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767084.043406:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767084.043409:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114633088:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767084.043415:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114633216:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767084.043434:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767084.043444:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767084/real 1775767084]  req@ffff9f14ab4ff100 x1862022114633216/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767109 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767084.043454:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff100 x1862022114633216/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767109 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767084.043462:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767084.043465:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767084.043468:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114633216:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767084.043480:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114633344:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767084.043496:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767084.043505:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767084/real 1775767084]  req@ffff9f14ab4fdc00 x1862022114633344/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767109 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767084.043515:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdc00 x1862022114633344/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767109 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767084.043523:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767084.043526:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767084.043529:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114633344:192.168.202.110@tcp:8:kworker.0
00010000:00010000:0.0:1775767084.298747:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767085.066660:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767085.322699:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767086.090627:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:0.0:1775767086.346706:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767087.114650:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767087.370705:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767088.138670:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775767088.394749:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767089.162630:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:1.0:1775767089.162632:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767089.162640:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1775767089.162644:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767089.162651:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4466
00000100:00080000:1.0:1775767089.162660:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767089.162667:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775767089.162699:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fed80 x1862022114633472/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767089.162720:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767089.162723:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775767089.162724:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767089.162728:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4466
00000100:00080000:1.0:1775767089.162731:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767089.162733:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767089.162741:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767089.162754:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fca80 x1862022114633600/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775767089.162758:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114633472:192.168.202.110@tcp:250:kworker.0
00000100:00080000:1.0:1775767089.162770:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767089.162774:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775767089.162776:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767089.162780:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4466
00000100:00080000:1.0:1775767089.162783:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767089.162786:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767089.162789:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767089.162799:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fce00 x1862022114633728/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767089.162805:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767089.162807:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775767089.162808:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767089.162811:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4466
00000100:00080000:1.0:1775767089.162813:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767089.162816:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767089.162819:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:0.0:1775767089.162824:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767089.162831:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fea00 x1862022114633856/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767089.162837:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767089.162839:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1775767089.162841:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767089.162844:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4466
00000100:00080000:1.0:1775767089.162849:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767089.162851:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767089.162855:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775767089.162863:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff800 x1862022114633984/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775767089.162871:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767089/real 1775767089]  req@ffff9f14ab4fed80 x1862022114633472/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767114 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767089.162871:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767089.162879:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775767089.162882:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767089.162890:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4466
00000100:00080000:1.0:1775767089.162893:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767089.162896:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767089.162899:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767089.162900:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fed80 x1862022114633472/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767114 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767089.162910:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdf80 x1862022114634112/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767089.162911:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767089.162920:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:1.0:1775767089.162923:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1775767089.162927:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114633472:192.168.202.110@tcp:250:kworker.0
00000100:00080000:1.0:1775767089.162928:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775767089.162946:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767089.162951:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4466
00000100:00080000:1.0:1775767089.162954:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767089.162958:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767089.162962:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767089.162968:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114633600:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775767089.162973:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ffb80 x1862022114634240/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767089.162981:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767089.162984:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775767089.162986:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767089.162991:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4466
00000800:00000100:0.0:1775767089.162994:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:1.0:1775767089.162996:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767089.163000:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767089.163004:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775767089.163011:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767089/real 1775767089]  req@ffff9f14ab4fca80 x1862022114633600/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767114 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767089.163015:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fed80 x1862022114634368/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775767089.163035:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fca80 x1862022114633600/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767114 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767089.163044:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767089.163048:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767089.163052:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114633600:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767089.163058:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114633728:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767089.163075:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767089.163088:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767089/real 1775767089]  req@ffff9f14ab4fce00 x1862022114633728/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767114 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767089.163097:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fce00 x1862022114633728/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767114 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767089.163102:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767089.163105:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767089.163107:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114633728:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767089.163111:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114633856:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767089.163124:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767089.163130:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767089/real 1775767089]  req@ffff9f14ab4fea00 x1862022114633856/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767114 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767089.163137:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fea00 x1862022114633856/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767114 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767089.163142:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767089.163144:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767089.163146:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114633856:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775767089.163150:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114633984:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767089.163159:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767089.163164:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767089/real 1775767089]  req@ffff9f14ab4ff800 x1862022114633984/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767114 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767089.163176:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff800 x1862022114633984/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767114 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767089.163190:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767089.163192:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767089.163193:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114633984:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775767089.163197:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114634112:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767089.163209:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767089.163215:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767089/real 1775767089]  req@ffff9f14ab4fdf80 x1862022114634112/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767114 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767089.163223:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdf80 x1862022114634112/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767114 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767089.163227:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767089.163229:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767089.163231:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114634112:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767089.163242:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ffb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114634240:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767089.163253:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767089.163265:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767089/real 1775767089]  req@ffff9f14ab4ffb80 x1862022114634240/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767114 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767089.163273:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ffb80 x1862022114634240/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767114 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767089.163282:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767089.163285:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767089.163287:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ffb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114634240:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767089.163290:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114634368:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767089.163300:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767089.163306:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767089/real 1775767089]  req@ffff9f14ab4fed80 x1862022114634368/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767114 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767089.163313:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fed80 x1862022114634368/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767114 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767089.163318:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767089.163320:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767089.163321:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114634368:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775767089.418715:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767090.186633:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767090.442755:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767091.210670:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:0.0:1775767091.466768:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767092.234638:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767092.490707:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767093.258642:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:0.0:1775767093.514725:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767094.282620:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:2.0:1775767094.282646:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767094.282653:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1775767094.282657:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767094.282665:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4471
00000100:00080000:2.0:1775767094.282676:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767094.282683:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775767094.282719:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0380 x1862022114634496/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767094.282748:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767094.282752:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775767094.282755:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767094.282760:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4471
00000100:00080000:2.0:1775767094.282766:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767094.282771:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775767094.282776:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114634496:192.168.202.110@tcp:250:kworker.0
00000008:00100000:2.0:1775767094.282783:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775767094.282801:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2d80 x1862022114634624/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767094.282821:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767094.282825:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775767094.282827:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000800:00000100:0.0:1775767094.282831:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:2.0:1775767094.282836:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4471
00000100:00080000:2.0:1775767094.282842:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767094.282847:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767094.282853:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775767094.282869:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0000 x1862022114634752/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775767094.282871:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767094/real 1775767094]  req@ffff9f14852d0380 x1862022114634496/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767119 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767094.282902:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767094.282910:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775767094.282915:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767094.282923:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4471
00000100:00080000:2.0:1775767094.282937:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767094.282943:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775767094.282948:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0380 x1862022114634496/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767119 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:2.0:1775767094.282952:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00080000:0.0:1775767094.282959:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767094.282963:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767094.282968:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114634496:192.168.202.110@tcp:250:kworker.0
00000100:00100000:2.0:1775767094.282973:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3b80 x1862022114634880/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767094.282985:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767094.282989:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00100000:0.0:1775767094.282991:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114634624:192.168.202.110@tcp:38:kworker.0
00000100:00080000:2.0:1775767094.282993:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767094.283000:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4471
00000100:00080000:2.0:1775767094.283006:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767094.283011:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000800:00000100:0.0:1775767094.283016:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000008:00100000:2.0:1775767094.283017:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775767094.283027:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767094/real 1775767094]  req@ffff9f14852d2d80 x1862022114634624/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767119 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767094.283035:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0e00 x1862022114635008/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775767094.283041:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2d80 x1862022114634624/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767119 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767094.283053:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767094.283059:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767094.283060:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775767094.283065:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00080000:2.0:1775767094.283065:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775767094.283072:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114634624:192.168.202.110@tcp:38:kworker.0
00000100:00080000:2.0:1775767094.283075:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4471
00000100:00100000:0.0:1775767094.283080:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114634752:192.168.202.110@tcp:38:kworker.0
00000100:00080000:2.0:1775767094.283083:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767094.283090:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000800:00000100:0.0:1775767094.283096:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000008:00100000:2.0:1775767094.283097:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767094.283105:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767094/real 1775767094]  req@ffff9f14852d0000 x1862022114634752/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767119 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767094.283115:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1880 x1862022114635136/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775767094.283116:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0000 x1862022114634752/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767119 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767094.283151:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767094.283159:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775767094.283164:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767094.283167:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767094.283171:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4471
00000100:00080000:0.0:1775767094.283173:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767094.283179:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114634752:192.168.202.110@tcp:38:kworker.0
00000100:00080000:2.0:1775767094.283180:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1775767094.283185:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114634880:192.168.202.110@tcp:8:kworker.0
00000100:00080000:2.0:1775767094.283189:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767094.283196:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000800:00000100:0.0:1775767094.283203:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767094.283212:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3100 x1862022114635264/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775767094.283213:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767094/real 1775767094]  req@ffff9f14852d3b80 x1862022114634880/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767119 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767094.283245:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3b80 x1862022114634880/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767119 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767094.283253:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767094.283257:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767094.283257:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775767094.283262:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767094.283264:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00080000:2.0:1775767094.283270:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4471
00000100:00100000:0.0:1775767094.283271:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114634880:192.168.202.110@tcp:8:kworker.0
00000100:00080000:2.0:1775767094.283277:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767094.283283:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767094.283287:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775767094.283290:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114635008:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775767094.283298:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2300 x1862022114635392/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1775767094.283307:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767094.283317:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767094/real 1775767094]  req@ffff9f14852d0e00 x1862022114635008/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767119 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767094.283325:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0e00 x1862022114635008/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767119 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767094.283330:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767094.283332:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767094.283335:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114635008:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775767094.283339:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114635136:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767094.283348:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767094.283361:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767094/real 1775767094]  req@ffff9f14852d1880 x1862022114635136/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767119 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767094.283368:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1880 x1862022114635136/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767119 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767094.283373:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767094.283376:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767094.283378:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114635136:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767094.283381:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114635264:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767094.283391:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767094.283396:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767094/real 1775767094]  req@ffff9f14852d3100 x1862022114635264/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767119 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767094.283403:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3100 x1862022114635264/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767119 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767094.283407:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767094.283409:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767094.283411:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114635264:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767094.283419:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114635392:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767094.283428:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767094.283434:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767094/real 1775767094]  req@ffff9f14852d2300 x1862022114635392/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767119 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767094.283447:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2300 x1862022114635392/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767119 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767094.283452:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767094.283454:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767094.283456:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114635392:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775767094.538744:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767095.306647:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767095.562726:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767096.330635:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767096.586767:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767097.354629:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767097.610759:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767098.378624:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:0.0:1775767098.634657:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1775767099.402634:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767099.402642:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1775767099.402646:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775767099.402647:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:2.0:1775767099.402656:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4476
00000100:00080000:2.0:1775767099.402665:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767099.402672:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775767099.402705:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3480 x1862022114635520/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767099.402726:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767099.402728:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775767099.402730:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767099.402733:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4476
00000100:00080000:2.0:1775767099.402735:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767099.402738:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767099.402747:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775767099.402756:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1c00 x1862022114635648/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767099.402764:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767099.402767:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775767099.402768:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775767099.402772:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114635520:192.168.202.110@tcp:250:kworker.0
00000100:00080000:2.0:1775767099.402774:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4476
00000100:00080000:2.0:1775767099.402788:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767099.402795:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767099.402800:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775767099.402815:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2300 x1862022114635776/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767099.402848:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000800:00000100:0.0:1775767099.402854:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:2.0:1775767099.402856:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775767099.402859:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767099.402863:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4476
00000100:00080000:2.0:1775767099.402868:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767099.402872:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767099.402877:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775767099.402890:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3100 x1862022114635904/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775767099.402895:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767099/real 1775767099]  req@ffff9f14852d3480 x1862022114635520/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767124 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767099.402923:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1775767099.402927:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3480 x1862022114635520/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767124 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767099.402928:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1775767099.402933:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767099.402940:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767099.402942:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4476
00000100:00080000:0.0:1775767099.402946:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:2.0:1775767099.402948:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1775767099.402951:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114635520:192.168.202.110@tcp:250:kworker.0
00000100:00080000:2.0:1775767099.402956:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767099.402961:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775767099.402972:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1880 x1862022114636032/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775767099.402975:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114635648:192.168.202.110@tcp:38:kworker.0
00000100:00080000:2.0:1775767099.402998:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767099.403001:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775767099.403003:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767099.403007:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4476
00000100:00080000:2.0:1775767099.403011:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000800:00000100:0.0:1775767099.403016:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:2.0:1775767099.403020:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767099.403024:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767099.403029:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767099/real 1775767099]  req@ffff9f14852d1c00 x1862022114635648/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767124 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767099.403036:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0e00 x1862022114636160/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775767099.403047:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1c00 x1862022114635648/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767124 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767099.403047:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767099.403052:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775767099.403054:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767099.403062:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4476
00000100:00080000:2.0:1775767099.403067:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767099.403071:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767099.403071:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767099.403074:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775767099.403080:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3b80 x1862022114636288/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767099.403089:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00080000:2.0:1775767099.403091:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767099.403095:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775767099.403097:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775767099.403098:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114635648:192.168.202.110@tcp:38:kworker.0
00000100:00080000:2.0:1775767099.403103:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4476
00000100:00080000:2.0:1775767099.403109:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767099.403111:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775767099.403115:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114635776:192.168.202.110@tcp:38:kworker.0
00000008:00100000:2.0:1775767099.403117:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775767099.403128:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0000 x1862022114636416/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1775767099.403144:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767099.403156:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767099/real 1775767099]  req@ffff9f14852d2300 x1862022114635776/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767124 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767099.403164:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2300 x1862022114635776/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767124 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767099.403169:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767099.403172:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767099.403174:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114635776:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767099.403178:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114635904:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767099.403188:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767099.403193:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767099/real 1775767099]  req@ffff9f14852d3100 x1862022114635904/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767124 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767099.403205:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3100 x1862022114635904/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767124 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767099.403210:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767099.403212:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767099.403214:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114635904:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775767099.403225:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114636032:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767099.403235:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767099.403240:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767099/real 1775767099]  req@ffff9f14852d1880 x1862022114636032/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767124 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767099.403248:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1880 x1862022114636032/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767124 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767099.403253:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767099.403254:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767099.403256:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114636032:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775767099.403259:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114636160:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767099.403269:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767099.403274:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767099/real 1775767099]  req@ffff9f14852d0e00 x1862022114636160/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767124 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767099.403281:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0e00 x1862022114636160/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767124 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767099.403291:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767099.403293:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767099.403295:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114636160:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767099.403298:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114636288:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767099.403306:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767099.403312:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767099/real 1775767099]  req@ffff9f14852d3b80 x1862022114636288/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767124 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767099.403318:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3b80 x1862022114636288/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767124 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767099.403323:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767099.403325:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767099.403327:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114636288:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767099.403330:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114636416:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767099.403339:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767099.403344:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767099/real 1775767099]  req@ffff9f14852d0000 x1862022114636416/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767124 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767099.403350:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0000 x1862022114636416/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767124 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767099.403355:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767099.403357:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767099.403359:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114636416:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775767099.658650:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767100.426687:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:0.0:1775767100.682678:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767101.450646:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767101.706752:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767102.474684:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767102.730745:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767103.498683:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767103.754819:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1775767104.522656:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1775767104.522659:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:2.0:1775767104.522668:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1775767104.522673:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767104.522682:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4482
00000100:00080000:2.0:1775767104.522694:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767104.522701:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775767104.522740:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2d80 x1862022114636544/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767104.522788:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767104.522792:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775767104.522795:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767104.522801:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4482
00000100:00080000:2.0:1775767104.522806:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1775767104.522810:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114636544:192.168.202.110@tcp:250:kworker.0
00000100:00080000:2.0:1775767104.522815:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767104.522826:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775767104.522842:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0380 x1862022114636672/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767104.522854:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767104.522859:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775767104.522860:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767104.522864:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4482
00000100:00080000:2.0:1775767104.522869:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000800:00000100:0.0:1775767104.522870:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:2.0:1775767104.522877:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767104.522884:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775767104.522897:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1500 x1862022114636800/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767104.522908:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767104.522911:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775767104.522913:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775767104.522918:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767104/real 1775767104]  req@ffff9f14852d2d80 x1862022114636544/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767129 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767104.522920:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4482
00000100:00080000:2.0:1775767104.522927:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767104.522935:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767104.522942:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775767104.522945:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2d80 x1862022114636544/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767129 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767104.522957:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00100000:2.0:1775767104.522958:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1f80 x1862022114636928/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767104.522967:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767104.522977:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114636544:192.168.202.110@tcp:250:kworker.0
00000100:00080000:2.0:1775767104.522978:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767104.522987:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1775767104.522990:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767104.522995:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4482
00000100:00080000:2.0:1775767104.523000:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767104.523004:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775767104.523008:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114636672:192.168.202.110@tcp:38:kworker.0
00000008:00100000:2.0:1775767104.523011:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775767104.523027:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0a80 x1862022114637056/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1775767104.523035:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:2.0:1775767104.523046:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1775767104.523051:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767104/real 1775767104]  req@ffff9f14852d0380 x1862022114636672/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767129 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767104.523051:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775767104.523056:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767104.523066:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4482
00000100:00100000:0.0:1775767104.523071:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0380 x1862022114636672/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767129 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767104.523073:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767104.523082:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767104.523083:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00080000:0.0:1775767104.523090:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000008:00100000:2.0:1775767104.523093:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767104.523094:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114636672:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767104.523103:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114636800:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767104.523113:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2680 x1862022114637184/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1775767104.523119:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767104.523128:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767104/real 1775767104]  req@ffff9f14852d1500 x1862022114636800/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767129 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767104.523151:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1775767104.523152:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1500 x1862022114636800/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767129 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767104.523158:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775767104.523163:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767104.523167:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767104.523171:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4482
00000100:00080000:0.0:1775767104.523173:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767104.523178:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114636800:192.168.202.110@tcp:38:kworker.0
00000100:00080000:2.0:1775767104.523180:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1775767104.523183:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114636928:192.168.202.110@tcp:8:kworker.0
00000100:00080000:2.0:1775767104.523189:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767104.523196:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000800:00000100:0.0:1775767104.523200:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767104.523208:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767104/real 1775767104]  req@ffff9f14852d1f80 x1862022114636928/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767129 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767104.523215:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1180 x1862022114637312/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775767104.523220:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1f80 x1862022114636928/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767129 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767104.523241:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767104.523243:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767104.523247:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775767104.523251:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00080000:2.0:1775767104.523252:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775767104.523259:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114636928:192.168.202.110@tcp:8:kworker.0
00000100:00080000:2.0:1775767104.523260:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4482
00000100:00080000:2.0:1775767104.523268:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767104.523272:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767104.523278:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775767104.523287:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114637056:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775767104.523292:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2a00 x1862022114637440/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1775767104.523310:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767104.523320:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767104/real 1775767104]  req@ffff9f14852d0a80 x1862022114637056/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767129 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767104.523329:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0a80 x1862022114637056/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767129 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767104.523343:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767104.523345:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767104.523347:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114637056:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775767104.523352:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114637184:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767104.523363:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767104.523370:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767104/real 1775767104]  req@ffff9f14852d2680 x1862022114637184/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767129 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767104.523377:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2680 x1862022114637184/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767129 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767104.523382:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767104.523384:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767104.523386:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114637184:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767104.523389:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114637312:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767104.523398:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767104.523404:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767104/real 1775767104]  req@ffff9f14852d1180 x1862022114637312/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767129 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767104.523415:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1180 x1862022114637312/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767129 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767104.523420:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767104.523422:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767104.523424:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114637312:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767104.523433:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114637440:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767104.523442:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767104.523447:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767104/real 1775767104]  req@ffff9f14852d2a00 x1862022114637440/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767129 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767104.523454:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2a00 x1862022114637440/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767129 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767104.523458:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767104.523460:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767104.523462:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114637440:192.168.202.110@tcp:8:kworker.0
00010000:00010000:3.0:1775767104.778681:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767105.546722:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:0.0:1775767105.802809:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767106.570702:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767106.826687:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767107.594694:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:0.0:1775767107.850718:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767108.618670:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767108.874788:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1775767109.642640:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1775767109.642641:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:2.0:1775767109.642651:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1775767109.642656:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767109.642667:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4487
00000100:00080000:2.0:1775767109.642678:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767109.642686:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775767109.642759:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3800 x1862022114637568/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767109.642789:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767109.642794:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775767109.642796:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767109.642800:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4487
00000100:00080000:2.0:1775767109.642805:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767109.642809:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767109.642819:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775767109.642836:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0700 x1862022114637696/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:1.0:1775767109.642845:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114637568:192.168.202.110@tcp:250:kworker.0
00000100:00080000:2.0:1775767109.642855:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767109.642859:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775767109.642861:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767109.642866:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4487
00000100:00080000:2.0:1775767109.642871:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767109.642875:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767109.642882:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775767109.642896:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2a00 x1862022114637824/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:1.0:1775767109.642903:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:2.0:1775767109.642920:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767109.642924:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775767109.642926:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767109.642932:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4487
00000100:00100000:1.0:1775767109.642937:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767109/real 1775767109]  req@ffff9f14852d3800 x1862022114637568/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767134 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767109.642937:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767109.642946:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767109.642954:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775767109.642956:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3800 x1862022114637568/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767134 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767109.642967:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767109.642970:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767109.642975:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1180 x1862022114637952/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:1.0:1775767109.642976:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114637568:192.168.202.110@tcp:250:kworker.0
00000100:00080000:2.0:1775767109.642992:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767109.642996:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1775767109.642998:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:1.0:1775767109.642999:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114637696:192.168.202.110@tcp:38:kworker.0
00000100:00080000:2.0:1775767109.643006:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4487
00000100:00080000:2.0:1775767109.643011:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767109.643016:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000800:00000100:1.0:1775767109.643021:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000008:00100000:2.0:1775767109.643026:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775767109.643031:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767109/real 1775767109]  req@ffff9f14852d0700 x1862022114637696/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767134 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767109.643044:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0700 x1862022114637696/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767134 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767109.643045:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2680 x1862022114638080/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767109.643078:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767109.643083:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00080000:2.0:1775767109.643083:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:1.0:1775767109.643087:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114637696:192.168.202.110@tcp:38:kworker.0
00000100:00080000:2.0:1775767109.643093:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00100000:1.0:1775767109.643096:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114637824:192.168.202.110@tcp:38:kworker.0
00000100:00080000:2.0:1775767109.643098:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767109.643106:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4487
00000800:00000100:1.0:1775767109.643111:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:2.0:1775767109.643112:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767109.643120:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775767109.643121:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767109/real 1775767109]  req@ffff9f14852d2a00 x1862022114637824/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767134 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:2.0:1775767109.643132:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767109.643134:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2a00 x1862022114637824/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767134 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767109.643143:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767109.643145:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767109.643148:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114637824:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767109.643153:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0a80 x1862022114638208/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:1.0:1775767109.643155:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114637952:192.168.202.110@tcp:8:kworker.0
00000100:00080000:2.0:1775767109.643172:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000800:00000100:1.0:1775767109.643174:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:2.0:1775767109.643179:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775767109.643182:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:1.0:1775767109.643183:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767109/real 1775767109]  req@ffff9f14852d1180 x1862022114637952/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767134 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767109.643190:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4487
00000100:00080000:2.0:1775767109.643199:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767109.643205:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775767109.643207:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1180 x1862022114637952/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767134 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:2.0:1775767109.643213:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00080000:1.0:1775767109.643222:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767109.643227:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767109.643232:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1f80 x1862022114638336/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:1.0:1775767109.643234:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114637952:192.168.202.110@tcp:8:kworker.0
00000100:00080000:2.0:1775767109.643251:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767109.643254:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775767109.643257:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:1.0:1775767109.643260:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114638080:192.168.202.110@tcp:8:kworker.0
00000100:00080000:2.0:1775767109.643265:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4487
00000100:00080000:2.0:1775767109.643271:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767109.643275:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767109.643282:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:1.0:1775767109.643284:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767109.643295:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767109/real 1775767109]  req@ffff9f14852d2680 x1862022114638080/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767134 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767109.643302:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1500 x1862022114638464/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:1.0:1775767109.643308:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2680 x1862022114638080/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767134 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767109.643328:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767109.643341:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767109.643344:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114638080:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775767109.643349:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114638208:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775767109.643362:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767109.643369:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767109/real 1775767109]  req@ffff9f14852d0a80 x1862022114638208/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767134 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767109.643381:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0a80 x1862022114638208/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767134 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767109.643389:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767109.643392:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767109.643396:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114638208:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775767109.643402:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114638336:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775767109.643421:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767109.643430:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767109/real 1775767109]  req@ffff9f14852d1f80 x1862022114638336/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767134 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767109.643438:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1f80 x1862022114638336/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767134 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767109.643443:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767109.643445:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767109.643447:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114638336:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775767109.643456:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114638464:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775767109.643467:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767109.643473:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767109/real 1775767109]  req@ffff9f14852d1500 x1862022114638464/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767134 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767109.643480:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1500 x1862022114638464/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767134 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767109.643485:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767109.643487:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767109.643489:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114638464:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775767109.898713:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767110.666606:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767110.922747:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767111.690688:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767111.946667:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767112.714720:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:0.0:1775767112.970714:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767113.738656:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767113.994724:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767114.762674:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:1.0:1775767114.762687:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767114.762696:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1775767114.762700:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767114.762709:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4492
00000100:00080000:1.0:1775767114.762722:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767114.762728:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775767114.762763:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdc00 x1862022114638592/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767114.762793:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767114.762797:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775767114.762798:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767114.762803:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4492
00000100:00080000:1.0:1775767114.762807:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767114.762811:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775767114.762819:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114638592:192.168.202.110@tcp:250:kworker.0
00000008:00100000:1.0:1775767114.762821:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767114.762837:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff100 x1862022114638720/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767114.762849:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767114.762853:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775767114.762855:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767114.762859:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4492
00000100:00080000:1.0:1775767114.762872:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767114.762877:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000800:00000100:0.0:1775767114.762881:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000008:00100000:1.0:1775767114.762886:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767114.762900:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fe300 x1862022114638848/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767114.762908:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767114.762911:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775767114.762912:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775767114.762916:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767114/real 1775767114]  req@ffff9f14ab4fdc00 x1862022114638592/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767139 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767114.762917:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4492
00000100:00080000:1.0:1775767114.762923:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767114.762930:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767114.762935:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775767114.762940:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdc00 x1862022114638592/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767139 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767114.762945:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fc380 x1862022114638976/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767114.762955:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767114.762958:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767114.762962:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:1.0:1775767114.762963:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1775767114.762967:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775767114.762969:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114638592:192.168.202.110@tcp:250:kworker.0
00000100:00080000:1.0:1775767114.762974:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4492
00000100:00080000:1.0:1775767114.762979:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767114.762983:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767114.762988:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775767114.762991:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114638720:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775767114.763006:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fd880 x1862022114639104/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1775767114.763014:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:1.0:1775767114.763025:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1775767114.763026:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767114/real 1775767114]  req@ffff9f14ab4ff100 x1862022114638720/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767139 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767114.763034:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775767114.763038:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767114.763045:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4492
00000100:00100000:0.0:1775767114.763049:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff100 x1862022114638720/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767139 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767114.763051:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767114.763058:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00080000:0.0:1775767114.763060:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767114.763064:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000008:00100000:1.0:1775767114.763064:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767114.763070:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114638720:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767114.763074:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114638848:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775767114.763080:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdc00 x1862022114639232/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767114.763089:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000800:00000100:0.0:1775767114.763091:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:1.0:1775767114.763094:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775767114.763097:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767114.763101:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4492
00000100:00080000:1.0:1775767114.763105:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767114.763108:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767114.763111:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767114.763112:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767114/real 1775767114]  req@ffff9f14ab4fe300 x1862022114638848/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767139 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767114.763122:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fed80 x1862022114639360/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775767114.763126:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fe300 x1862022114638848/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767139 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767114.763144:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767114.763145:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767114.763152:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775767114.763155:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00080000:1.0:1775767114.763155:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767114.763161:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4492
00000100:00100000:0.0:1775767114.763164:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114638848:192.168.202.110@tcp:38:kworker.0
00000100:00080000:1.0:1775767114.763165:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767114.763171:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767114.763175:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775767114.763179:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114638976:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775767114.763184:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ffb80 x1862022114639488/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1775767114.763208:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767114.763218:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767114/real 1775767114]  req@ffff9f14ab4fc380 x1862022114638976/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767139 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767114.763235:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fc380 x1862022114638976/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767139 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767114.763241:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767114.763243:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767114.763245:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114638976:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775767114.763256:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114639104:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767114.763268:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767114.763275:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767114/real 1775767114]  req@ffff9f14ab4fd880 x1862022114639104/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767139 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767114.763287:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fd880 x1862022114639104/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767139 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767114.763292:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767114.763293:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767114.763295:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114639104:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775767114.763299:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114639232:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767114.763308:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767114.763313:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767114/real 1775767114]  req@ffff9f14ab4fdc00 x1862022114639232/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767139 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767114.763321:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdc00 x1862022114639232/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767139 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767114.763325:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767114.763327:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767114.763329:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114639232:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767114.763332:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114639360:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767114.763341:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767114.763353:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767114/real 1775767114]  req@ffff9f14ab4fed80 x1862022114639360/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767139 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767114.763359:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fed80 x1862022114639360/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767139 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767114.763368:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767114.763370:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767114.763372:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114639360:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767114.763375:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ffb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114639488:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767114.763385:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767114.763391:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767114/real 1775767114]  req@ffff9f14ab4ffb80 x1862022114639488/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767139 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767114.763397:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ffb80 x1862022114639488/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767139 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767114.763402:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767114.763404:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767114.763406:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ffb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114639488:192.168.202.110@tcp:8:kworker.0
00010000:00010000:0.0:1775767115.018707:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767115.786702:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767116.042794:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767116.810668:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767117.066745:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767117.834650:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767118.090751:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767118.858674:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767119.114784:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767119.882656:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775767119.882700:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767119.882707:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775767119.882711:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767119.882721:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4497
00000100:00080000:3.0:1775767119.882732:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767119.882739:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775767119.882787:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744700 x1862022114639616/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767119.882816:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767119.882820:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775767119.882823:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767119.882828:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4497
00000100:00080000:3.0:1775767119.882840:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1775767119.882847:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114639616:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775767119.882848:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767119.882863:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767119.882882:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745500 x1862022114639744/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767119.882895:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767119.882899:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775767119.882901:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767119.882906:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4497
00000800:00000100:0.0:1775767119.882910:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775767119.882914:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767119.882921:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767119.882928:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767119.882946:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767119/real 1775767119]  req@ffff9f1485744700 x1862022114639616/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767144 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767119.882955:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745f80 x1862022114639872/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775767119.882988:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744700 x1862022114639616/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767144 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767119.883002:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767119.883003:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767119.883008:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767119.883013:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114639616:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775767119.883016:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775767119.883021:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767119.883028:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4497
00000100:00080000:3.0:1775767119.883033:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767119.883038:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775767119.883043:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114639744:192.168.202.110@tcp:38:kworker.0
00000008:00100000:3.0:1775767119.883046:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775767119.883063:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747800 x1862022114640000/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767119.883076:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000800:00000100:0.0:1775767119.883082:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775767119.883091:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775767119.883094:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775767119.883097:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767119/real 1775767119]  req@ffff9f1485745500 x1862022114639744/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767144 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767119.883102:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4497
00000100:00080000:3.0:1775767119.883110:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767119.883117:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775767119.883122:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745500 x1862022114639744/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767144 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:3.0:1775767119.883123:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00080000:0.0:1775767119.883137:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00100000:3.0:1775767119.883142:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744000 x1862022114640128/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767119.883147:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767119.883155:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114639744:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775767119.883167:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1775767119.883169:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114639872:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775767119.883172:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775767119.883174:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767119.883179:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4497
00000100:00080000:3.0:1775767119.883185:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767119.883191:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767119.883197:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767119.883210:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746680 x1862022114640256/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1775767119.883212:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775767119.883224:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767119.883228:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00100000:0.0:1775767119.883231:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767119/real 1775767119]  req@ffff9f1485745f80 x1862022114639872/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767144 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767119.883232:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767119.883240:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4497
00000100:00080000:3.0:1775767119.883247:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767119.883254:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775767119.883258:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745f80 x1862022114639872/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767144 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:3.0:1775767119.883260:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767119.883276:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745180 x1862022114640384/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767119.883284:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767119.883294:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00080000:3.0:1775767119.883296:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1775767119.883302:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114639872:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775767119.883302:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775767119.883306:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767119.883314:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4497
00000100:00080000:3.0:1775767119.883318:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767119.883322:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775767119.883326:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114640000:192.168.202.110@tcp:8:kworker.0
00000008:00100000:3.0:1775767119.883330:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775767119.883346:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745c00 x1862022114640512/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1775767119.883357:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767119.883374:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767119/real 1775767119]  req@ffff9f1485747800 x1862022114640000/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767144 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767119.883390:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747800 x1862022114640000/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767144 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767119.883399:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767119.883403:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767119.883406:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114640000:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775767119.883413:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114640128:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767119.883433:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767119.883444:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767119/real 1775767119]  req@ffff9f1485744000 x1862022114640128/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767144 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767119.883456:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744000 x1862022114640128/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767144 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767119.883464:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767119.883467:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767119.883471:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114640128:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775767119.883476:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114640256:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767119.883493:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767119.883503:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767119/real 1775767119]  req@ffff9f1485746680 x1862022114640256/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767144 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767119.883514:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746680 x1862022114640256/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767144 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767119.883522:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767119.883525:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767119.883528:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114640256:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767119.883534:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114640384:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767119.883552:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767119.883584:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767119/real 1775767119]  req@ffff9f1485745180 x1862022114640384/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767144 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767119.883596:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745180 x1862022114640384/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767144 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767119.883604:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767119.883607:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767119.883624:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114640384:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767119.883643:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114640512:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767119.883665:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767119.883678:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767119/real 1775767119]  req@ffff9f1485745c00 x1862022114640512/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767144 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767119.883691:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745c00 x1862022114640512/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767144 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767119.883699:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767119.883703:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767119.883706:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114640512:192.168.202.110@tcp:8:kworker.0
00010000:00010000:0.0:1775767120.138715:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767120.906688:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767121.162744:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767121.930683:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:0.0:1775767122.186660:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767122.954651:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767123.210706:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767123.978635:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767124.234728:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767125.002663:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:0.0:1775767125.002708:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767125.002713:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1775767125.002716:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767125.002723:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4502
00000100:00080000:0.0:1775767125.002738:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767125.002745:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775767125.002778:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3100 x1862022114640640/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767125.002802:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767125.002805:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775767125.002806:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767125.002810:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4502
00000100:00080000:0.0:1775767125.002813:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767125.002815:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767125.002823:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767125.002834:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114640640:192.168.202.110@tcp:250:kworker.0
00000100:00100000:0.0:1775767125.002844:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d2300 x1862022114640768/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767125.002857:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767125.002860:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:0.0:1775767125.002861:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767125.002866:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4502
00000100:00080000:0.0:1775767125.002872:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767125.002877:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767125.002880:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767125.002889:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0000 x1862022114640896/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767125.002894:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767125.002897:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775767125.002898:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767125.002904:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4502
00000800:00000100:3.0:1775767125.002904:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:0.0:1775767125.002909:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767125.002915:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767125.002919:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775767125.002925:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d1500 x1862022114641024/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767125.002931:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767125.002934:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1775767125.002935:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767125.002937:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4502
00000100:00080000:0.0:1775767125.002973:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767125.002976:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767125.002980:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775767125.002988:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3800 x1862022114641152/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767125.002995:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767125.002998:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775767125.003000:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767125.003003:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4502
00000100:00080000:0.0:1775767125.003005:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:3.0:1775767125.003006:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767125/real 1775767125]  req@ffff9f14a04d3100 x1862022114640640/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767150 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767125.003011:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767125.003017:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767125.003042:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d2a00 x1862022114641280/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767125.003049:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767125.003052:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:0.0:1775767125.003053:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767125.003055:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4502
00000100:00080000:0.0:1775767125.003059:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767125.003062:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767125.003066:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767125.003074:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0a80 x1862022114641408/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767125.003080:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767125.003082:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775767125.003083:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767125.003086:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4502
00000100:00080000:0.0:1775767125.003089:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767125.003092:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767125.003094:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775767125.003101:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3b80 x1862022114641536/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775767125.003146:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3100 x1862022114640640/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767150 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767125.003158:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767125.003163:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767125.003167:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114640640:192.168.202.110@tcp:250:kworker.0
00000100:00100000:3.0:1775767125.003193:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114640768:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775767125.003286:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767125.003298:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767125/real 1775767125]  req@ffff9f14a04d2300 x1862022114640768/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767150 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767125.003307:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d2300 x1862022114640768/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767150 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767125.003313:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767125.003315:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767125.003318:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114640768:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767125.003322:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114640896:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775767125.003336:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767125.003343:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767125/real 1775767125]  req@ffff9f14a04d0000 x1862022114640896/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767150 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767125.003350:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0000 x1862022114640896/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767150 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767125.003355:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767125.003358:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767125.003360:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114640896:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767125.003363:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114641024:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775767125.003373:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767125.003378:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767125/real 1775767125]  req@ffff9f14a04d1500 x1862022114641024/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767150 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767125.003385:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1500 x1862022114641024/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767150 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767125.003389:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767125.003391:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767125.003393:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114641024:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775767125.003395:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114641152:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775767125.003404:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767125.003409:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767125/real 1775767125]  req@ffff9f14a04d3800 x1862022114641152/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767150 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767125.003416:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3800 x1862022114641152/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767150 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767125.003422:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767125.003424:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767125.003425:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114641152:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775767125.003429:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114641280:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775767125.003437:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767125.003442:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767125/real 1775767125]  req@ffff9f14a04d2a00 x1862022114641280/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767150 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767125.003449:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d2a00 x1862022114641280/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767150 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767125.003454:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767125.003456:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767125.003458:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114641280:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767125.003461:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114641408:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775767125.003470:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767125.003475:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767125/real 1775767125]  req@ffff9f14a04d0a80 x1862022114641408/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767150 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767125.003481:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0a80 x1862022114641408/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767150 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767125.003486:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767125.003488:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767125.003490:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114641408:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767125.003493:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114641536:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775767125.003503:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767125.003526:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767125/real 1775767125]  req@ffff9f14a04d3b80 x1862022114641536/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767150 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767125.003534:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3b80 x1862022114641536/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767150 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767125.003540:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767125.003542:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767125.003545:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114641536:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775767125.258749:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767126.026722:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767126.282719:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767127.050620:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:0.0:1775767127.306692:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767128.074636:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767128.330769:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767129.098654:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:0.0:1775767129.354662:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767130.122660:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:1.0:1775767130.122660:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767130.122669:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1775767130.122673:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767130.122680:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4507
00000100:00080000:1.0:1775767130.122691:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767130.122698:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775767130.122732:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdf80 x1862022114641664/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767130.122755:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767130.122757:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775767130.122759:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767130.122761:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4507
00000100:00080000:1.0:1775767130.122764:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767130.122766:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767130.122774:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767130.122783:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff800 x1862022114641792/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767130.122791:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767130.122795:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00100000:3.0:1775767130.122796:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114641664:192.168.202.110@tcp:250:kworker.0
00000100:00080000:1.0:1775767130.122797:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767130.122811:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4507
00000100:00080000:1.0:1775767130.122814:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767130.122817:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767130.122820:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767130.122830:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fea00 x1862022114641920/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767130.122836:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767130.122839:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775767130.122840:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767130.122843:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4507
00000100:00080000:1.0:1775767130.122847:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767130.122849:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767130.122852:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775767130.122860:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fce00 x1862022114642048/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767130.122865:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767130.122868:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1775767130.122869:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767130.122871:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4507
00000100:00080000:1.0:1775767130.122873:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767130.122875:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767130.122878:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775767130.122885:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fca80 x1862022114642176/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767130.122894:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:3.0:1775767130.122894:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114641792:192.168.202.110@tcp:38:kworker.0
00000100:00080000:1.0:1775767130.122901:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775767130.122903:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767130.122906:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4507
00000100:00080000:1.0:1775767130.122909:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767130.122911:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767130.122916:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767130.122926:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ffb80 x1862022114642304/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775767130.122930:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114641920:192.168.202.110@tcp:38:kworker.0
00000100:00080000:1.0:1775767130.122940:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767130.122943:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775767130.122945:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767130.122950:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4507
00000100:00080000:1.0:1775767130.122953:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:3.0:1775767130.122954:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114642048:192.168.202.110@tcp:8:kworker.0
00000100:00080000:1.0:1775767130.122958:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767130.122963:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767130.122972:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fed80 x1862022114642432/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767130.122979:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767130.122982:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00100000:3.0:1775767130.122984:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114642176:192.168.202.110@tcp:8:kworker.0
00000100:00080000:1.0:1775767130.122985:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767130.122993:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4507
00000100:00080000:1.0:1775767130.122996:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767130.122998:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767130.123002:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775767130.123008:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ffb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114642304:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775767130.123015:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdc00 x1862022114642560/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775767130.123027:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114642432:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767130.123045:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114642560:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775767130.378934:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767131.146738:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767131.402735:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767132.170673:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767132.426687:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767133.194639:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000400:00000100:0.0:1775767133.194793:0:2393:0:(lib-socket.c:454:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.202.110:988
00000400:00000100:0.0:1775767133.194829:0:2393:0:(acceptor.c:111:lnet_connect_console_error()) Connection to 192.168.202.110@tcp at host 192.168.202.110 was unreachable: the network or that node may be down, or Lustre may be misconfigured.
00000800:00000100:0.0:1775767133.194846:0:2393:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:0.0:1775767133.194887:0:2393:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:0.0:1775767133.194897:0:2393:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:0.0:1775767133.194906:0:2393:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:0.0:1775767133.194917:0:2393:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:0.0:1775767133.194929:0:2393:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:0.0:1775767133.194939:0:2393:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:0.0:1775767133.194946:0:2393:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000100:00100000:3.0:1775767133.194955:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767130/real 1775767133]  req@ffff9f14ab4fca80 x1862022114642176/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767155 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767133.194983:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fca80 x1862022114642176/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767155 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767133.194997:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767133.195002:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767133.195007:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114642176:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775767133.195015:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767130/real 1775767133]  req@ffff9f14ab4ffb80 x1862022114642304/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767155 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767133.195029:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ffb80 x1862022114642304/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767155 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767133.195037:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767133.195041:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767133.195045:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ffb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114642304:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767133.195052:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767130/real 1775767133]  req@ffff9f14ab4fed80 x1862022114642432/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767155 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767133.195065:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fed80 x1862022114642432/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767155 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767133.195073:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767133.195076:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767133.195079:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114642432:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767133.195087:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767130/real 1775767133]  req@ffff9f14ab4ff800 x1862022114641792/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767155 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767133.195100:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff800 x1862022114641792/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767155 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767133.195108:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767133.195111:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767133.195114:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114641792:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767133.195121:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767130/real 1775767133]  req@ffff9f14ab4fea00 x1862022114641920/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767155 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767133.195133:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fea00 x1862022114641920/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767155 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767133.195141:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767133.195145:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767133.195148:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114641920:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767133.195155:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767130/real 1775767133]  req@ffff9f14ab4fce00 x1862022114642048/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767155 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767133.195165:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fce00 x1862022114642048/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767155 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767133.195173:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767133.195176:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767133.195179:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114642048:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775767133.195186:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767130/real 1775767133]  req@ffff9f14ab4fdf80 x1862022114641664/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767155 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767133.195197:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdf80 x1862022114641664/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767155 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767133.195205:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767133.195208:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767133.195211:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114641664:192.168.202.110@tcp:250:kworker.0
00000100:00100000:3.0:1775767133.195246:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767130/real 1775767133]  req@ffff9f14ab4fdc00 x1862022114642560/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767155 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767133.195259:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdc00 x1862022114642560/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767155 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767133.195267:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767133.195270:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767133.195273:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114642560:192.168.202.110@tcp:8:kworker.0
00010000:00010000:3.0:1775767133.450709:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767134.218666:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:0.0:1775767134.474682:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1775767135.242635:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767135.242643:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1775767135.242647:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00100000:2.0:1775767135.242647:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:1.0:1775767135.242656:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4512
00000100:00080000:1.0:1775767135.242666:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767135.242671:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775767135.242699:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fd880 x1862022114642688/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767135.242719:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767135.242722:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775767135.242723:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767135.242726:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4512
00000100:00080000:1.0:1775767135.242728:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767135.242730:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767135.242737:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767135.242745:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fc380 x1862022114642816/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767135.242751:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767135.242754:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775767135.242756:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767135.242760:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4512
00000100:00080000:1.0:1775767135.242763:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767135.242767:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767135.242773:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767135.242785:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114642688:192.168.202.110@tcp:250:kworker.0
00000100:00100000:1.0:1775767135.242786:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fe300 x1862022114642944/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767135.242811:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767135.242814:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775767135.242816:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767135.242821:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4512
00000100:00080000:1.0:1775767135.242825:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767135.242829:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767135.242835:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775767135.242849:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff100 x1862022114643072/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:3.0:1775767135.242855:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:1.0:1775767135.242865:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767135.242869:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1775767135.242871:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767135.242876:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4512
00000100:00080000:1.0:1775767135.242881:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767135.242885:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767135.242896:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775767135.242900:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767135/real 1775767135]  req@ffff9f14ab4fd880 x1862022114642688/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767160 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767135.242911:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdc00 x1862022114643200/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767135.242923:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:3.0:1775767135.242924:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fd880 x1862022114642688/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767160 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767135.242934:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775767135.242938:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767135.242947:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4512
00000100:00080000:1.0:1775767135.242957:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767135.242961:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767135.242966:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00080000:3.0:1775767135.242971:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000008:00100000:1.0:1775767135.242973:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767135.242977:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114642688:192.168.202.110@tcp:250:kworker.0
00000100:00100000:1.0:1775767135.242989:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdf80 x1862022114643328/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775767135.242990:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114642816:192.168.202.110@tcp:38:kworker.0
00000100:00080000:1.0:1775767135.243017:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767135.243021:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775767135.243023:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000800:00000100:3.0:1775767135.243029:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:1.0:1775767135.243033:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4512
00000100:00080000:1.0:1775767135.243040:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767135.243045:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767135.243051:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767135.243051:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767135/real 1775767135]  req@ffff9f14ab4fc380 x1862022114642816/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767160 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767135.243074:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fc380 x1862022114642816/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767160 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767135.243076:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fce00 x1862022114643456/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767135.243103:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767135.243105:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767135.243113:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775767135.243113:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00080000:1.0:1775767135.243118:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:3.0:1775767135.243121:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114642816:192.168.202.110@tcp:38:kworker.0
00000100:00080000:1.0:1775767135.243126:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4512
00000100:00080000:1.0:1775767135.243133:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767135.243138:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767135.243143:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775767135.243156:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114642944:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775767135.243158:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fea00 x1862022114643584/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:3.0:1775767135.243196:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767135.243211:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767135/real 1775767135]  req@ffff9f14ab4fe300 x1862022114642944/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767160 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767135.243224:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fe300 x1862022114642944/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767160 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767135.243231:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767135.243234:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767135.243238:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114642944:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767135.243244:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114643072:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775767135.243265:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767135.243276:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767135/real 1775767135]  req@ffff9f14ab4ff100 x1862022114643072/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767160 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767135.243287:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff100 x1862022114643072/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767160 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767135.243296:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767135.243300:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767135.243304:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114643072:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775767135.243309:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114643200:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775767135.243330:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767135.243341:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767135/real 1775767135]  req@ffff9f14ab4fdc00 x1862022114643200/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767160 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767135.243353:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdc00 x1862022114643200/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767160 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767135.243361:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767135.243365:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767135.243368:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114643200:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775767135.243374:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114643328:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775767135.243504:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767135.243516:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767135/real 1775767135]  req@ffff9f14ab4fdf80 x1862022114643328/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767160 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767135.243527:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdf80 x1862022114643328/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767160 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767135.243536:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767135.243539:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767135.243542:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114643328:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767135.243548:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114643456:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775767135.243595:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767135.243608:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767135/real 1775767135]  req@ffff9f14ab4fce00 x1862022114643456/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767160 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767135.243620:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fce00 x1862022114643456/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767160 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767135.243628:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767135.243632:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767135.243635:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114643456:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767135.243655:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114643584:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775767135.243675:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767135.243685:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767135/real 1775767135]  req@ffff9f14ab4fea00 x1862022114643584/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767160 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767135.243696:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fea00 x1862022114643584/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767160 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767135.243710:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767135.243714:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767135.243717:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114643584:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775767135.498687:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767136.266628:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:0.0:1775767136.522683:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767137.290632:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767137.546699:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767138.314694:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767138.570753:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767139.338679:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767139.594761:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767140.362633:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:1.0:1775767140.362648:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767140.362655:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1775767140.362660:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767140.362670:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4517
00000100:00080000:1.0:1775767140.362682:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767140.362692:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775767140.362741:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff800 x1862022114643712/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767140.362765:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767140.362780:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775767140.362781:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767140.362784:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4517
00000100:00080000:1.0:1775767140.362787:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767140.362789:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767140.362799:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767140.362809:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fed80 x1862022114643840/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767140.362819:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767140.362822:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775767140.362823:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767140.362829:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4517
00000100:00080000:1.0:1775767140.362835:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00100000:3.0:1775767140.362835:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114643712:192.168.202.110@tcp:250:kworker.0
00000100:00080000:1.0:1775767140.362844:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767140.362858:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767140.362877:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ffb80 x1862022114643968/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767140.362885:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767140.362887:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775767140.362888:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767140.362891:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4517
00000100:00080000:1.0:1775767140.362896:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767140.362899:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767140.362903:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775767140.362912:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fca80 x1862022114644096/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767140.362919:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767140.362922:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1775767140.362924:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767140.362927:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4517
00000100:00080000:1.0:1775767140.362929:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767140.362932:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767140.362937:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775767140.362945:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fea00 x1862022114644224/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:3.0:1775767140.362956:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:1.0:1775767140.362969:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767140.362972:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775767140.362975:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767140.362980:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4517
00000100:00080000:1.0:1775767140.362988:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767140.362993:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767140.362997:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767140.363013:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fce00 x1862022114644352/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775767140.363019:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767140/real 1775767140]  req@ffff9f14ab4ff800 x1862022114643712/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767165 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767140.363025:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767140.363036:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775767140.363037:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767140.363050:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4517
00000100:00080000:1.0:1775767140.363059:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767140.363072:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767140.363075:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767140.363084:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff800 x1862022114643712/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767165 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767140.363087:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdf80 x1862022114644480/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767140.363148:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767140.363148:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767140.363155:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775767140.363157:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767140.363161:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:1.0:1775767140.363168:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4517
00000100:00100000:3.0:1775767140.363170:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114643712:192.168.202.110@tcp:250:kworker.0
00000100:00080000:1.0:1775767140.363174:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767140.363189:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767140.363193:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775767140.363203:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdc00 x1862022114644608/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775767140.363212:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114643840:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775767140.363238:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767140.363247:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767140/real 1775767140]  req@ffff9f14ab4fed80 x1862022114643840/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767165 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767140.363256:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fed80 x1862022114643840/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767165 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767140.363262:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767140.363265:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767140.363268:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114643840:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767140.363273:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ffb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114643968:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775767140.363284:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767140.363289:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767140/real 1775767140]  req@ffff9f14ab4ffb80 x1862022114643968/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767165 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767140.363297:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ffb80 x1862022114643968/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767165 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767140.363302:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767140.363304:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767140.363306:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ffb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114643968:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767140.363310:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114644096:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775767140.363321:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767140.363326:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767140/real 1775767140]  req@ffff9f14ab4fca80 x1862022114644096/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767165 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767140.363334:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fca80 x1862022114644096/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767165 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767140.363339:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767140.363341:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767140.363343:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114644096:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775767140.363347:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114644224:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775767140.363357:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767140.363362:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767140/real 1775767140]  req@ffff9f14ab4fea00 x1862022114644224/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767165 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767140.363370:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fea00 x1862022114644224/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767165 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767140.363375:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767140.363377:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767140.363378:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114644224:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775767140.363382:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114644352:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775767140.363392:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767140.363430:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767140/real 1775767140]  req@ffff9f14ab4fce00 x1862022114644352/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767165 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767140.363442:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fce00 x1862022114644352/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767165 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767140.363448:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767140.363450:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767140.363452:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114644352:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767140.363457:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114644480:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775767140.363472:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767140.363479:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767140/real 1775767140]  req@ffff9f14ab4fdf80 x1862022114644480/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767165 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767140.363487:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdf80 x1862022114644480/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767165 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767140.363497:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767140.363499:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767140.363501:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114644480:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767140.363516:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114644608:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775767140.363526:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767140.363532:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767140/real 1775767140]  req@ffff9f14ab4fdc00 x1862022114644608/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767165 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767140.363539:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdc00 x1862022114644608/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767165 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767140.363544:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767140.363546:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767140.363548:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114644608:192.168.202.110@tcp:8:kworker.0
00010000:00010000:3.0:1775767140.618753:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767141.386666:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775767141.642768:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767142.410641:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767142.666737:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767143.434705:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:0.0:1775767143.690683:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767144.458686:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767144.714764:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1775767145.482760:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:2.0:1775767145.482767:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:0.0:1775767145.482771:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1775767145.482788:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767145.482804:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4522
00000100:00080000:0.0:1775767145.482818:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767145.482824:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775767145.482862:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0380 x1862022114644736/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767145.482889:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767145.482893:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775767145.482895:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767145.482901:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4522
00000100:00080000:0.0:1775767145.482906:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767145.482911:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767145.482923:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767145.482934:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114644736:192.168.202.110@tcp:250:kworker.0
00000100:00100000:0.0:1775767145.482942:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3480 x1862022114644864/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767145.482958:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767145.482962:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:0.0:1775767145.482964:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767145.482970:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4522
00000100:00080000:0.0:1775767145.482975:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767145.482979:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767145.482986:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000800:00000100:3.0:1775767145.482997:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767145.483002:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d1180 x1862022114644992/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767145.483016:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767145.483020:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775767145.483022:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767145.483026:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4522
00000100:00080000:0.0:1775767145.483030:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767145.483034:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767145.483040:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775767145.483047:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767145/real 1775767145]  req@ffff9f14a04d0380 x1862022114644736/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767170 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767145.483054:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0e00 x1862022114645120/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767145.483073:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767145.483076:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1775767145.483081:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:3.0:1775767145.483082:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0380 x1862022114644736/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767170 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767145.483086:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4522
00000100:00080000:0.0:1775767145.483092:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767145.483100:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767145.483109:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00080000:3.0:1775767145.483110:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767145.483120:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767145.483121:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d1f80 x1862022114645248/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767145.483133:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767145.483136:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775767145.483137:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:3.0:1775767145.483140:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114644736:192.168.202.110@tcp:250:kworker.0
00000100:00080000:0.0:1775767145.483142:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4522
00000100:00080000:0.0:1775767145.483148:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767145.483153:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767145.483156:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767145.483164:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3b80 x1862022114645376/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775767145.483171:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114644864:192.168.202.110@tcp:38:kworker.0
00000100:00080000:0.0:1775767145.483186:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767145.483189:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:0.0:1775767145.483191:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767145.483195:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4522
00000100:00080000:0.0:1775767145.483197:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767145.483201:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767145.483204:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000800:00000100:3.0:1775767145.483212:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767145.483216:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0a80 x1862022114645504/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767145.483225:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:3.0:1775767145.483229:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767145/real 1775767145]  req@ffff9f14a04d3480 x1862022114644864/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767170 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767145.483230:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775767145.483234:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767145.483242:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4522
00000100:00080000:0.0:1775767145.483249:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767145.483253:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767145.483256:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775767145.483257:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3480 x1862022114644864/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767170 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767145.483267:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d2a00 x1862022114645632/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767145.483268:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767145.483277:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767145.483285:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114644864:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767145.483293:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114644992:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775767145.483318:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767145.483325:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767145/real 1775767145]  req@ffff9f14a04d1180 x1862022114644992/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767170 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767145.483334:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1180 x1862022114644992/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767170 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767145.483340:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767145.483379:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767145.483383:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114644992:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767145.483388:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114645120:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775767145.483405:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767145.483414:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767145/real 1775767145]  req@ffff9f14a04d0e00 x1862022114645120/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767170 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767145.483423:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0e00 x1862022114645120/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767170 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767145.483443:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767145.483446:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767145.483448:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114645120:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775767145.483453:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114645248:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775767145.483465:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767145.483472:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767145/real 1775767145]  req@ffff9f14a04d1f80 x1862022114645248/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767170 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767145.483481:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1f80 x1862022114645248/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767170 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767145.483487:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767145.483489:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767145.483492:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114645248:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775767145.483505:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114645376:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775767145.483516:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767145.483522:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767145/real 1775767145]  req@ffff9f14a04d3b80 x1862022114645376/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767170 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767145.483530:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3b80 x1862022114645376/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767170 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767145.483537:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767145.483539:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767145.483542:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114645376:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767145.483547:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114645504:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775767145.483605:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767145.483618:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767145/real 1775767145]  req@ffff9f14a04d0a80 x1862022114645504/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767170 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767145.483650:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0a80 x1862022114645504/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767170 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767145.483661:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767145.483665:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767145.483669:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114645504:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767145.483677:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114645632:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775767145.483706:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767145.483719:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767145/real 1775767145]  req@ffff9f14a04d2a00 x1862022114645632/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767170 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767145.483733:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d2a00 x1862022114645632/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767170 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767145.483742:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767145.483745:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767145.483765:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114645632:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775767145.738718:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767146.506749:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767146.762736:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767147.530739:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767147.786893:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767148.554665:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775767148.810743:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767149.578709:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767149.834766:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767150.602656:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775767150.602716:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767150.602724:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775767150.602728:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767150.602739:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4528
00000100:00080000:3.0:1775767150.602752:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767150.602759:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775767150.602805:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747100 x1862022114645760/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767150.602841:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767150.602844:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775767150.602846:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767150.602850:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4528
00000100:00080000:3.0:1775767150.602852:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767150.602855:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767150.602864:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767150.602873:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744e00 x1862022114645888/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767150.602879:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767150.602882:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775767150.602884:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767150.602886:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4528
00000100:00080000:3.0:1775767150.602888:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767150.602891:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767150.602894:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767150.602901:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744380 x1862022114646016/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767150.602907:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767150.602910:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775767150.602911:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767150.602914:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4528
00000100:00080000:3.0:1775767150.602916:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767150.602919:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767150.602922:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775767150.602929:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114645760:192.168.202.110@tcp:250:kworker.0
00000100:00100000:3.0:1775767150.602930:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746a00 x1862022114646144/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767150.602942:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767150.602943:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775767150.602944:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767150.602946:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4528
00000100:00080000:3.0:1775767150.602948:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767150.602951:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767150.602954:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775767150.602963:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744a80 x1862022114646272/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767150.602969:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767150.602972:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775767150.602973:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767150.602976:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4528
00000100:00080000:3.0:1775767150.602981:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767150.602983:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767150.602986:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767150.602992:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747480 x1862022114646400/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767150.602998:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767150.603000:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775767150.603001:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767150.603004:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4528
00000100:00080000:3.0:1775767150.603007:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000800:00000100:0.0:1775767150.603010:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775767150.603014:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767150.603018:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767150.603025:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745880 x1862022114646528/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767150.603031:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767150.603034:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775767150.603036:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767150.603038:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4528
00000100:00080000:3.0:1775767150.603040:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767150.603042:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767150.603046:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775767150.603054:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746300 x1862022114646656/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775767150.603062:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767150/real 1775767150]  req@ffff9f1485747100 x1862022114645760/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767175 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767150.603090:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747100 x1862022114645760/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767175 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767150.603099:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767150.603103:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767150.603106:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114645760:192.168.202.110@tcp:250:kworker.0
00000100:00100000:0.0:1775767150.603112:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114645888:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767150.603128:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767150.603137:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767150/real 1775767150]  req@ffff9f1485744e00 x1862022114645888/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767175 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767150.603145:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744e00 x1862022114645888/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767175 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767150.603151:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767150.603153:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767150.603155:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114645888:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767150.603158:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114646016:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767150.603168:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767150.603189:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767150/real 1775767150]  req@ffff9f1485744380 x1862022114646016/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767175 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767150.603197:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744380 x1862022114646016/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767175 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767150.603202:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767150.603204:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767150.603206:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114646016:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767150.603229:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114646144:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767150.603243:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767150.603249:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767150/real 1775767150]  req@ffff9f1485746a00 x1862022114646144/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767175 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767150.603257:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746a00 x1862022114646144/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767175 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767150.603263:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767150.603265:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767150.603268:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114646144:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775767150.603271:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114646272:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767150.603280:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767150.603285:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767150/real 1775767150]  req@ffff9f1485744a80 x1862022114646272/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767175 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767150.603325:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744a80 x1862022114646272/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767175 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767150.603332:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767150.603334:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767150.603336:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114646272:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775767150.603340:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114646400:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767150.603355:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767150.603362:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767150/real 1775767150]  req@ffff9f1485747480 x1862022114646400/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767175 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767150.603369:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747480 x1862022114646400/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767175 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767150.603374:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767150.603376:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767150.603378:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114646400:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767150.603382:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114646528:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767150.603393:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767150.603398:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767150/real 1775767150]  req@ffff9f1485745880 x1862022114646528/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767175 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767150.603405:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745880 x1862022114646528/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767175 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767150.603410:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767150.603412:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767150.603413:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114646528:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767150.603416:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114646656:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767150.603426:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767150.603431:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767150/real 1775767150]  req@ffff9f1485746300 x1862022114646656/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767175 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767150.603438:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746300 x1862022114646656/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767175 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767150.603443:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767150.603445:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767150.603448:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114646656:192.168.202.110@tcp:8:kworker.0
00010000:00010000:0.0:1775767150.858696:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767151.626656:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767151.882726:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767152.650629:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767152.906725:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767153.674666:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767153.930798:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767154.698670:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767154.954818:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1775767155.722650:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767155.722661:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775767155.722666:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00100000:2.0:1775767155.722671:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775767155.722679:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4533
00000100:00080000:3.0:1775767155.722691:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767155.722699:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775767155.722736:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747b80 x1862022114646784/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767155.722767:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767155.722770:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775767155.722772:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767155.722776:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4533
00000100:00080000:3.0:1775767155.722780:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767155.722784:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767155.722794:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767155.722808:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746300 x1862022114646912/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767155.722819:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767155.722823:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775767155.722825:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767155.722830:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4533
00000100:00080000:3.0:1775767155.722833:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767155.722837:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767155.722842:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767155.722852:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745880 x1862022114647040/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767155.722863:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767155.722866:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775767155.722870:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775767155.722871:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114646784:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775767155.722876:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4533
00000100:00080000:3.0:1775767155.722884:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767155.722889:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767155.722894:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775767155.722907:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747480 x1862022114647168/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767155.722916:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767155.722920:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775767155.722922:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767155.722926:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4533
00000100:00080000:3.0:1775767155.722929:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767155.722933:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767155.722939:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:0.0:1775767155.722948:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767155.722952:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744a80 x1862022114647296/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767155.722964:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767155.722968:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775767155.722970:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767155.722973:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4533
00000100:00080000:3.0:1775767155.722978:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767155.722983:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767155.722988:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767155.722995:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767155/real 1775767155]  req@ffff9f1485747b80 x1862022114646784/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767180 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767155.723003:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746a00 x1862022114647424/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767155.723022:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767155.723026:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00100000:0.0:1775767155.723030:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747b80 x1862022114646784/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767180 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767155.723030:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767155.723040:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4533
00000100:00080000:0.0:1775767155.723047:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767155.723048:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767155.723054:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:3.0:1775767155.723056:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775767155.723061:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114646784:192.168.202.110@tcp:250:kworker.0
00000008:00100000:3.0:1775767155.723065:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767155.723069:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114646912:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767155.723084:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744380 x1862022114647552/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1775767155.723095:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775767155.723095:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767155.723101:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775767155.723103:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775767155.723109:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767155/real 1775767155]  req@ffff9f1485746300 x1862022114646912/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767180 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767155.723110:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4533
00000100:00080000:3.0:1775767155.723118:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1775767155.723123:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746300 x1862022114646912/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767180 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767155.723126:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767155.723137:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00080000:0.0:1775767155.723138:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767155.723144:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767155.723148:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114646912:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767155.723157:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744e00 x1862022114647680/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775767155.723170:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114647040:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767155.723184:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767155.723193:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767155/real 1775767155]  req@ffff9f1485745880 x1862022114647040/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767180 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767155.723206:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745880 x1862022114647040/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767180 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767155.723211:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767155.723212:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767155.723214:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114647040:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767155.723218:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114647168:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767155.723228:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767155.723234:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767155/real 1775767155]  req@ffff9f1485747480 x1862022114647168/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767180 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767155.723261:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747480 x1862022114647168/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767180 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767155.723269:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767155.723273:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767155.723276:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114647168:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775767155.723280:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114647296:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767155.723295:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767155.723301:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767155/real 1775767155]  req@ffff9f1485744a80 x1862022114647296/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767180 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767155.723309:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744a80 x1862022114647296/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767180 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767155.723318:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767155.723320:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767155.723322:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114647296:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775767155.723325:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114647424:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767155.723334:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767155.723340:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767155/real 1775767155]  req@ffff9f1485746a00 x1862022114647424/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767180 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767155.723346:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746a00 x1862022114647424/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767180 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767155.723351:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767155.723353:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767155.723355:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114647424:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767155.723358:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114647552:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767155.723368:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767155.723373:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767155/real 1775767155]  req@ffff9f1485744380 x1862022114647552/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767180 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767155.723379:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744380 x1862022114647552/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767180 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767155.723384:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767155.723386:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767155.723388:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114647552:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767155.723399:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114647680:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767155.723409:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767155.723415:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767155/real 1775767155]  req@ffff9f1485744e00 x1862022114647680/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767180 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767155.723422:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744e00 x1862022114647680/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767180 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767155.723427:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767155.723428:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767155.723430:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114647680:192.168.202.110@tcp:8:kworker.0
00010000:00010000:1.0:1775767155.978726:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767156.746680:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767157.002773:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767157.770679:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:0.0:1775767158.026682:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767158.794641:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767159.050759:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767159.818705:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767160.074747:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767160.842615:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:2.0:1775767160.842665:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767160.842670:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1775767160.842673:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767160.842683:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4538
00000100:00080000:2.0:1775767160.842697:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767160.842705:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775767160.842749:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0380 x1862022114647808/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767160.842780:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767160.842784:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775767160.842785:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767160.842789:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4538
00000100:00080000:2.0:1775767160.842793:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767160.842798:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767160.842811:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775767160.842832:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2d80 x1862022114647936/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775767160.842840:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114647808:192.168.202.110@tcp:250:kworker.0
00000100:00080000:2.0:1775767160.842857:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767160.842863:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775767160.842864:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767160.842869:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4538
00000100:00080000:2.0:1775767160.842873:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767160.842876:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767160.842882:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775767160.842893:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0000 x1862022114648064/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767160.842901:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767160.842903:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775767160.842905:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767160.842908:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4538
00000100:00080000:2.0:1775767160.842957:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767160.842961:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767160.842964:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775767160.842981:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3b80 x1862022114648192/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:3.0:1775767160.842985:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:2.0:1775767160.842998:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767160.843002:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1775767160.843004:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767160.843007:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4538
00000100:00080000:2.0:1775767160.843011:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767160.843014:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767160.843018:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775767160.843026:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0e00 x1862022114648320/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767160.843034:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767160.843037:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775767160.843039:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767160.843043:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4538
00000100:00080000:2.0:1775767160.843046:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767160.843048:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767160.843051:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775767160.843057:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1880 x1862022114648448/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767160.843063:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767160.843066:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775767160.843067:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767160.843070:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4538
00000100:00080000:2.0:1775767160.843072:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767160.843074:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767160.843076:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775767160.843082:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3100 x1862022114648576/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767160.843087:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767160.843089:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775767160.843089:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767160.843092:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4538
00000100:00080000:2.0:1775767160.843094:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767160.843096:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767160.843100:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775767160.843106:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2300 x1862022114648704/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775767160.843149:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767160/real 1775767160]  req@ffff9f14852d0380 x1862022114647808/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767185 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767160.843225:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0380 x1862022114647808/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767185 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767160.843237:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767160.843241:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767160.843246:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114647808:192.168.202.110@tcp:250:kworker.0
00000100:00100000:3.0:1775767160.843280:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114647936:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775767160.843305:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767160.843315:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767160/real 1775767160]  req@ffff9f14852d2d80 x1862022114647936/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767185 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767160.843323:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2d80 x1862022114647936/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767185 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767160.843329:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767160.843333:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767160.843337:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114647936:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767160.843343:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114648064:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775767160.843364:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767160.843375:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767160/real 1775767160]  req@ffff9f14852d0000 x1862022114648064/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767185 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767160.843388:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0000 x1862022114648064/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767185 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767160.843398:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767160.843400:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767160.843402:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114648064:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767160.843407:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114648192:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775767160.843424:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767160.843431:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767160/real 1775767160]  req@ffff9f14852d3b80 x1862022114648192/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767185 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767160.843439:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3b80 x1862022114648192/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767185 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767160.843444:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767160.843446:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767160.843448:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114648192:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775767160.843452:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114648320:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775767160.843462:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767160.843468:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767160/real 1775767160]  req@ffff9f14852d0e00 x1862022114648320/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767185 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767160.843475:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0e00 x1862022114648320/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767185 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767160.843480:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767160.843482:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767160.843483:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114648320:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775767160.843488:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114648448:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775767160.843498:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767160.843503:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767160/real 1775767160]  req@ffff9f14852d1880 x1862022114648448/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767185 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767160.843511:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1880 x1862022114648448/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767185 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767160.843516:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767160.843518:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767160.843520:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114648448:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767160.843523:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114648576:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775767160.843533:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767160.843539:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767160/real 1775767160]  req@ffff9f14852d3100 x1862022114648576/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767185 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767160.843546:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3100 x1862022114648576/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767185 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767160.843578:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767160.843581:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767160.843585:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114648576:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767160.843597:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114648704:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775767160.843617:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767160.843634:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767160/real 1775767160]  req@ffff9f14852d2300 x1862022114648704/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767185 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767160.843650:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2300 x1862022114648704/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767185 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767160.843656:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767160.843658:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767160.843660:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114648704:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775767161.098716:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767161.866680:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767162.122702:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767162.890654:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775767163.146754:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767163.914638:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767164.170689:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767164.938672:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:0.0:1775767165.194658:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1775767165.962652:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767165.962660:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00100000:2.0:1775767165.962662:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:0.0:1775767165.962664:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767165.962675:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4543
00000100:00080000:0.0:1775767165.962685:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767165.962691:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775767165.962723:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3800 x1862022114648832/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767165.962747:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767165.962749:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775767165.962751:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767165.962754:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4543
00000100:00080000:0.0:1775767165.962757:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767165.962759:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767165.962767:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767165.962775:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d1500 x1862022114648960/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767165.962782:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767165.962785:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:0.0:1775767165.962786:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767165.962789:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4543
00000100:00080000:0.0:1775767165.962791:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00100000:3.0:1775767165.962794:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114648832:192.168.202.110@tcp:250:kworker.0
00000100:00080000:0.0:1775767165.962796:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767165.962804:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767165.962812:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0000 x1862022114649088/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767165.962819:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767165.962821:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775767165.962822:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767165.962825:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4543
00000100:00080000:0.0:1775767165.962827:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767165.962829:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767165.962833:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775767165.962839:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d2300 x1862022114649216/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767165.962846:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767165.962848:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1775767165.962849:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767165.962851:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4543
00000100:00080000:0.0:1775767165.962854:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000800:00000100:3.0:1775767165.962858:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:0.0:1775767165.962860:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767165.962865:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775767165.962873:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3100 x1862022114649344/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767165.962879:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767165.962881:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775767165.962882:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767165.962885:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4543
00000100:00080000:0.0:1775767165.962887:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767165.962890:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767165.962894:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767165.962898:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767165/real 1775767165]  req@ffff9f14a04d3800 x1862022114648832/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767190 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767165.962904:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d2a00 x1862022114649472/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767165.962919:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767165.962921:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:0.0:1775767165.962922:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:3.0:1775767165.962926:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3800 x1862022114648832/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767190 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767165.962927:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4543
00000100:00080000:0.0:1775767165.962932:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767165.962939:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767165.962944:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00080000:3.0:1775767165.962945:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767165.962953:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767165.962957:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0a80 x1862022114649600/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775767165.962961:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114648832:192.168.202.110@tcp:250:kworker.0
00000100:00080000:0.0:1775767165.962974:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:3.0:1775767165.962974:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114648960:192.168.202.110@tcp:38:kworker.0
00000100:00080000:0.0:1775767165.962982:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775767165.962984:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767165.962987:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4543
00000100:00080000:0.0:1775767165.962989:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767165.962992:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767165.962996:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:3.0:1775767165.963000:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767165.963006:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3b80 x1862022114649728/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775767165.963023:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767165/real 1775767165]  req@ffff9f14a04d1500 x1862022114648960/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767190 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767165.963032:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1500 x1862022114648960/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767190 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767165.963038:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767165.963041:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767165.963043:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114648960:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767165.963060:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114649088:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775767165.963072:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767165.963095:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767165/real 1775767165]  req@ffff9f14a04d0000 x1862022114649088/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767190 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767165.963104:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0000 x1862022114649088/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767190 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767165.963110:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767165.963112:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767165.963114:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114649088:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767165.963118:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114649216:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775767165.963165:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767165.963174:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767165/real 1775767165]  req@ffff9f14a04d2300 x1862022114649216/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767190 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767165.963182:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d2300 x1862022114649216/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767190 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767165.963187:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767165.963189:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767165.963191:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114649216:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775767165.963195:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114649344:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775767165.963207:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767165.963213:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767165/real 1775767165]  req@ffff9f14a04d3100 x1862022114649344/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767190 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767165.963220:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3100 x1862022114649344/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767190 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767165.963225:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767165.963226:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767165.963228:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114649344:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775767165.963231:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114649472:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775767165.963240:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767165.963246:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767165/real 1775767165]  req@ffff9f14a04d2a00 x1862022114649472/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767190 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767165.963258:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d2a00 x1862022114649472/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767190 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767165.963266:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767165.963269:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767165.963273:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114649472:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767165.963278:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114649600:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775767165.963299:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767165.963309:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767165/real 1775767165]  req@ffff9f14a04d0a80 x1862022114649600/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767190 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767165.963318:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0a80 x1862022114649600/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767190 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767165.963323:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767165.963326:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767165.963328:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114649600:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767165.963332:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114649728:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775767165.963345:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767165.963353:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767165/real 1775767165]  req@ffff9f14a04d3b80 x1862022114649728/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767190 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767165.963365:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3b80 x1862022114649728/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767190 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767165.963372:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767165.963375:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767165.963379:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114649728:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775767166.218678:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767166.986688:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767167.242708:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767168.010698:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767168.266673:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767169.034627:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767169.290732:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767170.058653:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775767170.314723:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1775767171.082709:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1775767171.082715:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:1.0:1775767171.082721:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1775767171.082726:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767171.082740:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4548
00000100:00080000:1.0:1775767171.082753:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767171.082760:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775767171.082809:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff100 x1862022114649856/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767171.082837:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767171.082840:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775767171.082842:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767171.082845:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4548
00000100:00080000:1.0:1775767171.082848:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767171.082851:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767171.082862:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767171.082874:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fe300 x1862022114649984/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767171.082883:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767171.082887:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775767171.082889:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767171.082892:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4548
00000100:00080000:1.0:1775767171.082895:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767171.082898:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767171.082902:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767171.082905:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114649856:192.168.202.110@tcp:250:kworker.0
00000100:00100000:1.0:1775767171.082912:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fc380 x1862022114650112/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767171.082921:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767171.082924:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775767171.082925:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767171.082928:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4548
00000100:00080000:1.0:1775767171.082930:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767171.082932:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767171.082936:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775767171.082945:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fd880 x1862022114650240/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767171.082952:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767171.082954:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1775767171.082956:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767171.082958:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4548
00000100:00080000:1.0:1775767171.082962:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767171.082965:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767171.082970:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775767171.082983:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdc00 x1862022114650368/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767171.082995:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000800:00000100:3.0:1775767171.082997:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:1.0:1775767171.083003:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775767171.083005:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767171.083012:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4548
00000100:00080000:1.0:1775767171.083018:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767171.083025:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767171.083047:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767171.083063:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdf80 x1862022114650496/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767171.083073:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767171.083093:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00100000:3.0:1775767171.083094:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767171/real 1775767171]  req@ffff9f14ab4ff100 x1862022114649856/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767196 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767171.083095:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767171.083104:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4548
00000100:00080000:1.0:1775767171.083110:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767171.083133:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767171.083139:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767171.083154:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fce00 x1862022114650624/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775767171.083154:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff100 x1862022114649856/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767196 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767171.083180:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767171.083189:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767171.083194:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114649856:192.168.202.110@tcp:250:kworker.0
00000100:00080000:1.0:1775767171.083196:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767171.083203:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00100000:3.0:1775767171.083205:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114649984:192.168.202.110@tcp:38:kworker.0
00000100:00080000:1.0:1775767171.083207:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767171.083215:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4548
00000100:00080000:1.0:1775767171.083221:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767171.083226:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767171.083234:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:3.0:1775767171.083234:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767171.083248:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767171/real 1775767171]  req@ffff9f14ab4fe300 x1862022114649984/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767196 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767171.083254:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fea00 x1862022114650752/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775767171.083273:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fe300 x1862022114649984/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767196 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767171.083280:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767171.083283:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767171.083286:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114649984:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767171.083309:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114650112:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775767171.083325:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767171.083332:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767171/real 1775767171]  req@ffff9f14ab4fc380 x1862022114650112/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767196 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767171.083341:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fc380 x1862022114650112/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767196 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767171.083347:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767171.083349:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767171.083351:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114650112:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767171.083355:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114650240:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775767171.083366:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767171.083371:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767171/real 1775767171]  req@ffff9f14ab4fd880 x1862022114650240/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767196 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767171.083379:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fd880 x1862022114650240/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767196 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767171.083385:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767171.083387:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767171.083388:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114650240:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775767171.083408:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114650368:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775767171.083429:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767171.083440:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767171/real 1775767171]  req@ffff9f14ab4fdc00 x1862022114650368/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767196 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767171.083464:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdc00 x1862022114650368/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767196 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767171.083474:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767171.083478:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767171.083481:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114650368:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775767171.083486:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114650496:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775767171.083504:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767171.083512:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767171/real 1775767171]  req@ffff9f14ab4fdf80 x1862022114650496/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767196 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767171.083522:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdf80 x1862022114650496/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767196 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767171.083528:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767171.083530:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767171.083532:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114650496:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767171.083536:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114650624:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775767171.083547:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767171.083553:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767171/real 1775767171]  req@ffff9f14ab4fce00 x1862022114650624/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767196 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767171.083561:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fce00 x1862022114650624/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767196 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767171.083571:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767171.083574:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767171.083576:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114650624:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767171.083580:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114650752:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775767171.083618:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767171.083626:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767171/real 1775767171]  req@ffff9f14ab4fea00 x1862022114650752/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767196 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767171.083634:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fea00 x1862022114650752/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767196 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767171.083641:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767171.083643:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767171.083646:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114650752:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775767171.338777:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767172.106747:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767172.362687:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767173.130692:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767173.386753:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767174.154697:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767174.410773:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767175.178648:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767175.434733:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767176.202647:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:1.0:1775767176.202650:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767176.202658:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1775767176.202662:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767176.202672:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4553
00000100:00080000:1.0:1775767176.202684:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767176.202692:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775767176.202736:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fca80 x1862022114650880/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767176.202766:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767176.202771:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775767176.202773:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767176.202777:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4553
00000100:00080000:1.0:1775767176.202782:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767176.202786:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767176.202796:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767176.202810:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ffb80 x1862022114651008/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775767176.202815:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114650880:192.168.202.110@tcp:250:kworker.0
00000100:00080000:1.0:1775767176.202835:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767176.202838:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775767176.202840:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767176.202845:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4553
00000100:00080000:1.0:1775767176.202849:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767176.202852:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767176.202857:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767176.202868:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fed80 x1862022114651136/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767176.202877:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767176.202881:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775767176.202883:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767176.202888:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4553
00000800:00000100:3.0:1775767176.202888:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:1.0:1775767176.202895:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767176.202899:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767176.202903:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775767176.202914:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff800 x1862022114651264/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767176.202924:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767176.202927:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1775767176.202929:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767176.202933:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4553
00000100:00080000:1.0:1775767176.202938:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00100000:3.0:1775767176.202938:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767176/real 1775767176]  req@ffff9f14ab4fca80 x1862022114650880/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767201 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767176.202944:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767176.202951:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775767176.202966:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fea00 x1862022114651392/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775767176.202971:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fca80 x1862022114650880/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767201 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767176.202987:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767176.202991:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775767176.202992:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767176.202993:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767176.202998:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:1.0:1775767176.203001:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4553
00000100:00080000:1.0:1775767176.203006:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:3.0:1775767176.203006:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114650880:192.168.202.110@tcp:250:kworker.0
00000100:00080000:1.0:1775767176.203013:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767176.203018:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767176.203048:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fce00 x1862022114651520/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767176.203057:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767176.203061:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775767176.203064:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:3.0:1775767176.203065:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ffb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114651008:192.168.202.110@tcp:38:kworker.0
00000100:00080000:1.0:1775767176.203071:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4553
00000100:00080000:1.0:1775767176.203076:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767176.203080:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767176.203085:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000800:00000100:3.0:1775767176.203099:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767176.203100:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdf80 x1862022114651648/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775767176.203113:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767176/real 1775767176]  req@ffff9f14ab4ffb80 x1862022114651008/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767201 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767176.203130:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767176.203135:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775767176.203138:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767176.203144:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4553
00000100:00100000:3.0:1775767176.203147:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ffb80 x1862022114651008/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767201 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767176.203149:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767176.203156:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767176.203166:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00080000:3.0:1775767176.203169:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767176.203176:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767176.203183:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdc00 x1862022114651776/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775767176.203183:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ffb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114651008:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767176.203198:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114651136:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775767176.203220:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767176.203234:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767176/real 1775767176]  req@ffff9f14ab4fed80 x1862022114651136/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767201 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767176.203248:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fed80 x1862022114651136/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767201 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767176.203256:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767176.203260:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767176.203263:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114651136:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767176.203268:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114651264:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775767176.203284:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767176.203292:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767176/real 1775767176]  req@ffff9f14ab4ff800 x1862022114651264/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767201 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767176.203313:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff800 x1862022114651264/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767201 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767176.203319:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767176.203321:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767176.203323:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114651264:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775767176.203327:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114651392:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775767176.203340:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767176.203350:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767176/real 1775767176]  req@ffff9f14ab4fea00 x1862022114651392/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767201 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767176.203363:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fea00 x1862022114651392/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767201 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767176.203371:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767176.203374:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767176.203378:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114651392:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775767176.203384:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114651520:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775767176.203405:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767176.203416:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767176/real 1775767176]  req@ffff9f14ab4fce00 x1862022114651520/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767201 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767176.203429:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fce00 x1862022114651520/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767201 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767176.203439:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767176.203442:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767176.203446:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114651520:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767176.203466:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114651648:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775767176.203488:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767176.203499:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767176/real 1775767176]  req@ffff9f14ab4fdf80 x1862022114651648/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767201 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767176.203512:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdf80 x1862022114651648/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767201 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767176.203520:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767176.203523:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767176.203525:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114651648:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767176.203530:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114651776:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775767176.203545:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767176.203586:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767176/real 1775767176]  req@ffff9f14ab4fdc00 x1862022114651776/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767201 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767176.203596:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdc00 x1862022114651776/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767201 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767176.203602:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767176.203604:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767176.203607:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114651776:192.168.202.110@tcp:8:kworker.0
00010000:00010000:3.0:1775767176.458695:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767177.226669:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767177.482753:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767178.250710:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767178.506798:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767179.274651:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767179.530740:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767180.298657:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767180.554680:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767181.322704:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:2.0:1775767181.322710:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767181.322717:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1775767181.322722:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767181.322732:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4558
00000100:00080000:2.0:1775767181.322744:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767181.322751:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775767181.322790:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1c00 x1862022114651904/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767181.322817:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767181.322821:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775767181.322823:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767181.322827:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4558
00000100:00080000:2.0:1775767181.322831:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767181.322837:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767181.322847:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767181.322864:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114651904:192.168.202.110@tcp:250:kworker.0
00000100:00100000:2.0:1775767181.322874:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3480 x1862022114652032/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767181.322884:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767181.322887:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775767181.322888:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767181.322891:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4558
00000100:00080000:2.0:1775767181.322894:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767181.322897:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767181.322901:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775767181.322910:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2300 x1862022114652160/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767181.322917:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767181.322919:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775767181.322920:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767181.322922:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4558
00000100:00080000:2.0:1775767181.322925:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000800:00000100:3.0:1775767181.322929:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:2.0:1775767181.322932:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767181.322936:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775767181.322943:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3100 x1862022114652288/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767181.322949:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767181.322952:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1775767181.322953:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767181.322956:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4558
00000100:00080000:2.0:1775767181.322959:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767181.322962:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767181.322965:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775767181.322989:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767181/real 1775767181]  req@ffff9f14852d1c00 x1862022114651904/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767206 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767181.322992:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1880 x1862022114652416/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767181.323047:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767181.323050:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00100000:3.0:1775767181.323053:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1c00 x1862022114651904/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767206 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767181.323054:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767181.323064:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4558
00000100:00080000:3.0:1775767181.323070:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767181.323072:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767181.323076:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:2.0:1775767181.323081:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775767181.323082:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114651904:192.168.202.110@tcp:250:kworker.0
00000008:00100000:2.0:1775767181.323092:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767181.323106:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114652032:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767181.323110:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0e00 x1862022114652544/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767181.323125:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767181.323128:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775767181.323133:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000800:00000100:3.0:1775767181.323133:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:2.0:1775767181.323139:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4558
00000100:00100000:3.0:1775767181.323171:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767181/real 1775767181]  req@ffff9f14852d3480 x1862022114652032/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767206 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767181.323173:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:3.0:1775767181.323185:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3480 x1862022114652032/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767206 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767181.323186:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767181.323197:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00080000:3.0:1775767181.323197:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767181.323204:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767181.323207:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114652032:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767181.323213:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114652160:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767181.323217:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3b80 x1862022114652672/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767181.323229:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000800:00000100:3.0:1775767181.323250:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:2.0:1775767181.323255:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775767181.323260:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:3.0:1775767181.323270:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767181/real 1775767181]  req@ffff9f14852d2300 x1862022114652160/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767206 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767181.323271:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4558
00000100:00080000:2.0:1775767181.323282:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767181.323293:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767181.323304:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775767181.323311:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2300 x1862022114652160/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767206 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767181.323342:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0000 x1862022114652800/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767181.323342:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767181.323353:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767181.323361:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114652160:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767181.323372:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114652288:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775767181.323392:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767181.323400:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767181/real 1775767181]  req@ffff9f14852d3100 x1862022114652288/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767206 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767181.323408:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3100 x1862022114652288/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767206 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767181.323415:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767181.323419:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767181.323422:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114652288:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775767181.323443:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114652416:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775767181.323462:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767181.323472:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767181/real 1775767181]  req@ffff9f14852d1880 x1862022114652416/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767206 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767181.323493:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1880 x1862022114652416/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767206 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767181.323517:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767181.323520:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767181.323522:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114652416:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775767181.323537:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114652544:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775767181.323553:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767181.323560:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767181/real 1775767181]  req@ffff9f14852d0e00 x1862022114652544/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767206 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767181.323568:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0e00 x1862022114652544/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767206 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767181.323573:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767181.323576:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767181.323577:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114652544:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767181.323581:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114652672:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775767181.323591:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767181.323596:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767181/real 1775767181]  req@ffff9f14852d3b80 x1862022114652672/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767206 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767181.323603:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3b80 x1862022114652672/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767206 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767181.323613:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767181.323615:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767181.323617:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114652672:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767181.323620:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114652800:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775767181.323653:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767181.323664:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767181/real 1775767181]  req@ffff9f14852d0000 x1862022114652800/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767206 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767181.323676:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0000 x1862022114652800/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767206 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767181.323701:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767181.323704:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767181.323708:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114652800:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775767181.578741:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767182.346674:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767182.602683:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767183.370630:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767183.626752:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767184.394674:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767184.650789:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767185.418632:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767185.674725:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1775767186.442688:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:2.0:1775767186.442688:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:0.0:1775767186.442698:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1775767186.442703:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767186.442713:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4563
00000100:00080000:0.0:1775767186.442725:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767186.442733:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775767186.442775:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d1f80 x1862022114652928/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767186.442805:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767186.442810:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775767186.442812:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767186.442816:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4563
00000100:00080000:0.0:1775767186.442820:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767186.442825:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767186.442835:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767186.442850:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0e00 x1862022114653056/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775767186.442856:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114652928:192.168.202.110@tcp:250:kworker.0
00000100:00080000:0.0:1775767186.442874:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767186.442879:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:0.0:1775767186.442881:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767186.442887:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4563
00000100:00080000:0.0:1775767186.442892:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767186.442898:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767186.442904:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767186.442935:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d1180 x1862022114653184/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:3.0:1775767186.442940:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:0.0:1775767186.442968:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767186.442972:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775767186.442975:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767186.442982:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4563
00000100:00080000:0.0:1775767186.442988:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767186.442994:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767186.443002:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775767186.443009:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767186/real 1775767186]  req@ffff9f14a04d1f80 x1862022114652928/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767211 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767186.443020:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3480 x1862022114653312/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775767186.443032:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1f80 x1862022114652928/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767211 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767186.443038:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767186.443047:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775767186.443047:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767186.443050:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767186.443054:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:0.0:1775767186.443063:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4563
00000100:00100000:3.0:1775767186.443063:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114652928:192.168.202.110@tcp:250:kworker.0
00000100:00080000:0.0:1775767186.443075:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767186.443083:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767186.443089:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775767186.443097:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114653056:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767186.443106:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0380 x1862022114653440/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767186.443143:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000800:00000100:3.0:1775767186.443146:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:0.0:1775767186.443151:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775767186.443155:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767186.443161:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4563
00000100:00080000:0.0:1775767186.443166:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767186.443170:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767186.443176:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767186.443179:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767186/real 1775767186]  req@ffff9f14a04d0e00 x1862022114653056/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767211 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767186.443192:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d1f80 x1862022114653568/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775767186.443201:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0e00 x1862022114653056/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767211 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767186.443209:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767186.443219:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:0.0:1775767186.443222:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767186.443223:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767186.443228:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4563
00000100:00080000:3.0:1775767186.443230:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767186.443236:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114653056:192.168.202.110@tcp:38:kworker.0
00000100:00080000:0.0:1775767186.443237:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767186.443247:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775767186.443247:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114653184:192.168.202.110@tcp:38:kworker.0
00000008:00100000:0.0:1775767186.443258:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767186.443275:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3b80 x1862022114653696/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:3.0:1775767186.443279:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:0.0:1775767186.443291:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:3.0:1775767186.443291:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767186/real 1775767186]  req@ffff9f14a04d1180 x1862022114653184/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767211 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767186.443302:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775767186.443308:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:3.0:1775767186.443312:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1180 x1862022114653184/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767211 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767186.443317:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4563
00000100:00080000:0.0:1775767186.443328:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767186.443328:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767186.443333:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767186.443336:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114653184:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767186.443341:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114653312:192.168.202.110@tcp:8:kworker.0
00000100:00080000:0.0:1775767186.443350:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767186.443357:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:3.0:1775767186.443357:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767186.443369:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767186/real 1775767186]  req@ffff9f14a04d3480 x1862022114653312/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767211 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767186.443376:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0a80 x1862022114653824/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775767186.443394:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3480 x1862022114653312/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767211 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767186.443404:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767186.443408:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767186.443413:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114653312:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775767186.443432:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114653440:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775767186.443449:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767186.443457:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767186/real 1775767186]  req@ffff9f14a04d0380 x1862022114653440/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767211 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767186.443465:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0380 x1862022114653440/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767211 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767186.443471:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767186.443473:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767186.443475:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114653440:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775767186.443478:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114653568:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775767186.443489:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767186.443494:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767186/real 1775767186]  req@ffff9f14a04d1f80 x1862022114653568/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767211 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767186.443506:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1f80 x1862022114653568/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767211 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767186.443511:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767186.443513:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767186.443515:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114653568:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767186.443518:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114653696:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775767186.443537:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767186.443546:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767186/real 1775767186]  req@ffff9f14a04d3b80 x1862022114653696/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767211 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767186.443596:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3b80 x1862022114653696/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767211 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767186.443606:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767186.443608:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767186.443610:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114653696:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767186.443625:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114653824:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775767186.443650:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767186.443659:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767186/real 1775767186]  req@ffff9f14a04d0a80 x1862022114653824/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767211 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767186.443667:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0a80 x1862022114653824/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767211 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767186.443677:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767186.443679:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767186.443681:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114653824:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775767186.698684:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767187.466630:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767187.722697:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767188.490656:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767188.746730:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775767189.514711:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767189.770757:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775767190.538687:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767190.794724:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1775767191.562656:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:3.0:1775767191.562659:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:0.0:1775767191.562670:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1775767191.562676:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767191.562689:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4569
00000100:00080000:0.0:1775767191.562703:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767191.562711:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775767191.562757:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d2a00 x1862022114653952/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767191.562788:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767191.562792:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775767191.562794:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767191.562799:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4569
00000100:00080000:0.0:1775767191.562804:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767191.562809:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767191.562822:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767191.562833:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114653952:192.168.202.110@tcp:250:kworker.0
00000100:00100000:0.0:1775767191.562840:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3100 x1862022114654080/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767191.562853:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767191.562876:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:0.0:1775767191.562877:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767191.562881:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4569
00000100:00080000:0.0:1775767191.562885:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767191.562888:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767191.562893:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767191.562907:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d2300 x1862022114654208/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767191.562919:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767191.562922:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775767191.562924:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767191.562928:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4569
00000800:00000100:3.0:1775767191.562933:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:0.0:1775767191.562934:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767191.562941:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767191.562947:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775767191.562961:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0000 x1862022114654336/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767191.562970:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767191.562973:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1775767191.562975:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767191.562980:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4569
00000100:00080000:0.0:1775767191.562983:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00100000:3.0:1775767191.562985:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767191/real 1775767191]  req@ffff9f14a04d2a00 x1862022114653952/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767216 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767191.562986:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767191.562993:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775767191.563007:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d1500 x1862022114654464/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775767191.563013:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d2a00 x1862022114653952/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767216 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767191.563024:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767191.563026:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767191.563031:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775767191.563031:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:0.0:1775767191.563035:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:3.0:1775767191.563038:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114653952:192.168.202.110@tcp:250:kworker.0
00000100:00080000:0.0:1775767191.563041:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4569
00000100:00080000:0.0:1775767191.563046:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767191.563050:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767191.563054:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767191.563062:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3800 x1862022114654592/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775767191.563066:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114654080:192.168.202.110@tcp:38:kworker.0
00000100:00080000:0.0:1775767191.563079:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767191.563086:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:0.0:1775767191.563087:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767191.563091:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4569
00000800:00000100:3.0:1775767191.563095:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:0.0:1775767191.563097:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767191.563102:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767191.563107:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767191.563110:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767191/real 1775767191]  req@ffff9f14a04d3100 x1862022114654080/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767216 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767191.563118:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d2a00 x1862022114654720/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775767191.563125:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3100 x1862022114654080/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767216 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767191.563131:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767191.563139:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775767191.563142:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767191.563150:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4569
00000100:00080000:0.0:1775767191.563155:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767191.563159:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00080000:3.0:1775767191.563159:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000008:00100000:0.0:1775767191.563166:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00080000:3.0:1775767191.563168:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767191.563172:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114654080:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767191.563176:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0a80 x1862022114654848/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775767191.563182:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114654208:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775767191.563205:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767191.563212:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767191/real 1775767191]  req@ffff9f14a04d2300 x1862022114654208/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767216 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767191.563222:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d2300 x1862022114654208/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767216 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767191.563227:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767191.563230:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767191.563232:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114654208:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767191.563237:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114654336:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775767191.563253:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767191.563261:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767191/real 1775767191]  req@ffff9f14a04d0000 x1862022114654336/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767216 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767191.563270:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0000 x1862022114654336/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767216 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767191.563276:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767191.563278:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767191.563281:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114654336:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775767191.563285:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114654464:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775767191.563295:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767191.563300:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767191/real 1775767191]  req@ffff9f14a04d1500 x1862022114654464/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767216 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767191.563308:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1500 x1862022114654464/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767216 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767191.563313:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767191.563315:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767191.563317:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114654464:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775767191.563328:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114654592:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775767191.563338:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767191.563344:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767191/real 1775767191]  req@ffff9f14a04d3800 x1862022114654592/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767216 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767191.563352:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3800 x1862022114654592/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767216 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767191.563357:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767191.563359:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767191.563360:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114654592:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767191.563364:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114654720:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775767191.563390:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767191.563397:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767191/real 1775767191]  req@ffff9f14a04d2a00 x1862022114654720/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767216 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767191.563405:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d2a00 x1862022114654720/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767216 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767191.563410:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767191.563412:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767191.563413:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114654720:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767191.563417:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114654848:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775767191.563426:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767191.563431:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767191/real 1775767191]  req@ffff9f14a04d0a80 x1862022114654848/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767216 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767191.563438:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0a80 x1862022114654848/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767216 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767191.563443:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767191.563445:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767191.563447:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114654848:192.168.202.110@tcp:8:kworker.0
00010000:00010000:3.0:1775767191.818698:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775767192.586666:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767192.842748:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775767193.610646:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767193.866764:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775767194.634651:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767194.890758:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775767195.658638:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767195.914712:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775767196.682679:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:2.0:1775767196.682694:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767196.682727:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1775767196.682732:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767196.682745:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4574
00000100:00080000:2.0:1775767196.682760:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767196.682770:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775767196.682846:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2d80 x1862022114654976/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767196.682905:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767196.682910:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775767196.682911:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767196.682917:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4574
00000100:00080000:2.0:1775767196.682923:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767196.682929:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767196.682942:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767196.682944:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114654976:192.168.202.110@tcp:250:kworker.0
00000100:00100000:2.0:1775767196.682966:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0380 x1862022114655104/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767196.682982:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767196.682987:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775767196.682990:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767196.682996:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4574
00000100:00080000:2.0:1775767196.683002:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767196.683023:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767196.683031:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775767196.683042:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1500 x1862022114655232/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767196.683051:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767196.683054:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775767196.683056:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767196.683060:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4574
00000100:00080000:2.0:1775767196.683063:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767196.683070:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767196.683075:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775767196.683083:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1f80 x1862022114655360/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775767196.683090:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114655104:192.168.202.110@tcp:38:kworker.0
00000100:00080000:2.0:1775767196.683093:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767196.683098:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1775767196.683101:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767196.683108:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4574
00000100:00080000:2.0:1775767196.683116:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767196.683121:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775767196.683127:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114655232:192.168.202.110@tcp:38:kworker.0
00000008:00100000:2.0:1775767196.683128:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775767196.683149:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0a80 x1862022114655488/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767196.683161:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1775767196.683168:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114655360:192.168.202.110@tcp:8:kworker.0
00000100:00080000:2.0:1775767196.683168:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775767196.683172:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767196.683179:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4574
00000100:00080000:2.0:1775767196.683184:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767196.683192:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767196.683200:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767196.683230:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114655488:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775767196.683230:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2680 x1862022114655616/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767196.683248:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767196.683252:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00100000:0.0:1775767196.683253:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114655616:192.168.202.110@tcp:38:kworker.0
00000100:00080000:2.0:1775767196.683255:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767196.683274:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4574
00000100:00080000:2.0:1775767196.683279:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767196.683282:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767196.683288:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775767196.683301:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1180 x1862022114655744/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767196.683331:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767196.683334:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775767196.683336:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767196.683339:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4574
00000100:00080000:2.0:1775767196.683341:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767196.683344:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767196.683347:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775767196.683355:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2a00 x1862022114655872/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:1.0:1775767196.683401:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114655744:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775767196.683496:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114655872:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775767196.938724:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775767197.706682:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767197.962771:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775767198.730690:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767198.986744:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775767199.754663:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000400:00000100:2.0:1775767199.754912:0:2391:0:(lib-socket.c:454:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.202.110:988
00000400:00000100:2.0:1775767199.754977:0:2391:0:(acceptor.c:111:lnet_connect_console_error()) Connection to 192.168.202.110@tcp at host 192.168.202.110 was unreachable: the network or that node may be down, or Lustre may be misconfigured.
00000800:00000100:2.0:1775767199.755001:0:2391:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:2.0:1775767199.755055:0:2391:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:2.0:1775767199.755070:0:2391:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:2.0:1775767199.755083:0:2391:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:2.0:1775767199.755095:0:2391:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000100:00100000:1.0:1775767199.755097:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767196/real 1775767199]  req@ffff9f14852d0380 x1862022114655104/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767221 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000800:00000100:2.0:1775767199.755108:0:2391:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:2.0:1775767199.755124:0:2391:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000100:00100000:1.0:1775767199.755147:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0380 x1862022114655104/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767221 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000800:00000100:2.0:1775767199.755149:0:2391:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000100:00080000:1.0:1775767199.755170:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767199.755179:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767199.755185:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114655104:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775767199.755194:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767196/real 1775767199]  req@ffff9f14852d1500 x1862022114655232/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767221 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767199.755219:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1500 x1862022114655232/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767221 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767199.755228:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767199.755232:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767199.755236:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114655232:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775767199.755246:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767196/real 1775767199]  req@ffff9f14852d2d80 x1862022114654976/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767221 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767199.755264:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2d80 x1862022114654976/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767221 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767199.755272:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767199.755276:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767199.755280:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114654976:192.168.202.110@tcp:250:kworker.0
00000100:00100000:1.0:1775767199.755326:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767196/real 1775767199]  req@ffff9f14852d1180 x1862022114655744/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767221 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767199.755340:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1180 x1862022114655744/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767221 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767199.755347:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767199.755350:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767199.755354:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114655744:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775767199.755361:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767196/real 1775767199]  req@ffff9f14852d2a00 x1862022114655872/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767221 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767199.755375:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2a00 x1862022114655872/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767221 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767199.755384:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767199.755388:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767199.755391:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114655872:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775767199.755398:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767196/real 1775767199]  req@ffff9f14852d2680 x1862022114655616/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767221 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767199.755412:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2680 x1862022114655616/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767221 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767199.755420:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767199.755424:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767199.755428:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114655616:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775767199.755438:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767196/real 1775767199]  req@ffff9f14852d1f80 x1862022114655360/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767221 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767199.755452:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1f80 x1862022114655360/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767221 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767199.755461:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767199.755464:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767199.755468:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114655360:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775767199.755477:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767196/real 1775767199]  req@ffff9f14852d0a80 x1862022114655488/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767221 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767199.755492:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0a80 x1862022114655488/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767221 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767199.755501:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767199.755506:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767199.755510:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114655488:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775767200.010720:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775767200.778627:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767201.034743:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775767201.802678:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:2.0:1775767201.802701:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767201.802712:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1775767201.802718:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767201.802729:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4579
00000100:00080000:2.0:1775767201.802741:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767201.802748:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775767201.802809:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0700 x1862022114656000/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767201.802840:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767201.802843:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775767201.802845:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767201.802849:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4579
00000100:00080000:2.0:1775767201.802853:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767201.802858:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767201.802869:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775767201.802880:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3800 x1862022114656128/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:1.0:1775767201.802891:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114656000:192.168.202.110@tcp:250:kworker.0
00000100:00080000:2.0:1775767201.802911:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767201.802916:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775767201.802918:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767201.802925:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4579
00000100:00080000:2.0:1775767201.802931:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767201.802936:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767201.802942:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775767201.802960:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0a80 x1862022114656256/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767201.802972:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767201.802976:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775767201.802979:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000800:00000100:1.0:1775767201.802990:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:2.0:1775767201.802994:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4579
00000100:00080000:2.0:1775767201.803002:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767201.803009:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767201.803022:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775767201.803047:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1f80 x1862022114656384/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:1.0:1775767201.803058:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767201/real 1775767201]  req@ffff9f14852d0700 x1862022114656000/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767226 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767201.803079:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767201.803093:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1775767201.803098:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767201.803111:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4579
00000100:00080000:2.0:1775767201.803120:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00100000:1.0:1775767201.803126:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0700 x1862022114656000/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767226 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767201.803138:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767201.803150:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00080000:1.0:1775767201.803152:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00100000:2.0:1775767201.803168:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2680 x1862022114656512/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767201.803181:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:2.0:1775767201.803181:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767201.803189:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00100000:1.0:1775767201.803191:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114656000:192.168.202.110@tcp:250:kworker.0
00000100:00080000:2.0:1775767201.803193:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767201.803207:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4579
00000100:00080000:2.0:1775767201.803214:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767201.803220:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767201.803228:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767201.803239:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114656128:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767201.803248:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2a00 x1862022114656640/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767201.803271:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000800:00000100:1.0:1775767201.803280:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:2.0:1775767201.803284:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775767201.803288:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767201.803298:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4579
00000100:00080000:2.0:1775767201.803307:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:1.0:1775767201.803315:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767201/real 1775767201]  req@ffff9f14852d3800 x1862022114656128/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767226 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767201.803320:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767201.803332:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767201.803346:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3800 x1862022114656128/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767226 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767201.803361:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1180 x1862022114656768/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767201.803387:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767201.803394:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767201.803401:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00080000:2.0:1775767201.803403:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775767201.803411:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:1.0:1775767201.803415:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114656128:192.168.202.110@tcp:38:kworker.0
00000100:00080000:2.0:1775767201.803423:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4579
00000100:00080000:2.0:1775767201.803431:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:1.0:1775767201.803435:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114656256:192.168.202.110@tcp:38:kworker.0
00000100:00080000:2.0:1775767201.803440:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767201.803448:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775767201.803467:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2d80 x1862022114656896/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:1.0:1775767201.803510:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767201.803529:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767201/real 1775767201]  req@ffff9f14852d0a80 x1862022114656256/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767226 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767201.803544:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0a80 x1862022114656256/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767226 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767201.803581:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767201.803586:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767201.803590:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114656256:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775767201.803599:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114656384:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775767201.803627:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767201.803640:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767201/real 1775767201]  req@ffff9f14852d1f80 x1862022114656384/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767226 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767201.803653:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1f80 x1862022114656384/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767226 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767201.803661:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767201.803664:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767201.803668:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114656384:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775767201.803675:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114656512:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775767201.803695:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767201.803707:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767201/real 1775767201]  req@ffff9f14852d2680 x1862022114656512/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767226 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767201.803726:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2680 x1862022114656512/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767226 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767201.803734:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767201.803737:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767201.803740:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114656512:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775767201.803761:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114656640:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775767201.803798:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767201.803810:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767201/real 1775767201]  req@ffff9f14852d2a00 x1862022114656640/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767226 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767201.803823:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2a00 x1862022114656640/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767226 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767201.803831:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767201.803834:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767201.803837:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114656640:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775767201.803845:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114656768:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775767201.803865:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767201.803877:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767201/real 1775767201]  req@ffff9f14852d1180 x1862022114656768/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767226 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767201.803888:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1180 x1862022114656768/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767226 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767201.803895:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767201.803898:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767201.803902:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114656768:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775767201.803908:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114656896:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775767201.803927:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767201.803937:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767201/real 1775767201]  req@ffff9f14852d2d80 x1862022114656896/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767226 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767201.803947:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2d80 x1862022114656896/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767226 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767201.803954:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767201.803957:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767201.803960:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114656896:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775767202.058745:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775767202.826618:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767203.082797:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775767203.850612:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767204.106840:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775767204.874610:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767205.130728:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775767205.898623:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767206.154908:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1775767206.922652:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:3.0:1775767206.922656:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:2.0:1775767206.922665:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1775767206.922670:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767206.922681:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4584
00000100:00080000:2.0:1775767206.922694:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767206.922702:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775767206.922766:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1500 x1862022114657024/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767206.922797:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767206.922800:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775767206.922802:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767206.922805:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4584
00000100:00080000:2.0:1775767206.922809:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767206.922812:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767206.922823:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775767206.922835:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0380 x1862022114657152/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767206.922844:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767206.922847:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775767206.922849:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:1.0:1775767206.922854:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114657024:192.168.202.110@tcp:250:kworker.0
00000100:00080000:2.0:1775767206.922855:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4584
00000100:00080000:2.0:1775767206.922860:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767206.922864:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767206.922867:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775767206.922875:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0000 x1862022114657280/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767206.922888:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767206.922891:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775767206.922893:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767206.922896:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4584
00000100:00080000:2.0:1775767206.922899:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767206.922903:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767206.922906:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775767206.922914:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3b80 x1862022114657408/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767206.922921:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767206.922924:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1775767206.922925:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767206.922927:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4584
00000100:00080000:2.0:1775767206.922930:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000800:00000100:1.0:1775767206.922934:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:2.0:1775767206.922935:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767206.922940:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775767206.922947:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0e00 x1862022114657536/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767206.922953:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767206.922956:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775767206.922958:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767206.922960:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4584
00000100:00080000:2.0:1775767206.922963:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767206.922966:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767206.922969:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775767206.922978:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1880 x1862022114657664/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:1.0:1775767206.922982:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767206/real 1775767206]  req@ffff9f14852d1500 x1862022114657024/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767231 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767206.923011:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767206.923014:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775767206.923017:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:1.0:1775767206.923018:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1500 x1862022114657024/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767231 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767206.923023:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4584
00000100:00080000:2.0:1775767206.923028:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767206.923035:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767206.923039:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775767206.923045:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3100 x1862022114657792/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767206.923048:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767206.923056:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767206.923057:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:2.0:1775767206.923063:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00100000:1.0:1775767206.923066:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114657024:192.168.202.110@tcp:250:kworker.0
00000100:00080000:2.0:1775767206.923066:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767206.923074:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4584
00000100:00080000:2.0:1775767206.923077:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767206.923079:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767206.923082:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775767206.923088:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2300 x1862022114657920/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:1.0:1775767206.923095:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114657152:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775767206.923117:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767206.923127:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767206/real 1775767206]  req@ffff9f14852d0380 x1862022114657152/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767231 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767206.923136:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0380 x1862022114657152/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767231 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767206.923142:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767206.923144:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767206.923146:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114657152:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775767206.923150:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114657280:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775767206.923161:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767206.923174:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767206/real 1775767206]  req@ffff9f14852d0000 x1862022114657280/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767231 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767206.923182:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0000 x1862022114657280/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767231 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767206.923188:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767206.923190:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767206.923192:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114657280:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775767206.923197:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114657408:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775767206.923210:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767206.923216:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767206/real 1775767206]  req@ffff9f14852d3b80 x1862022114657408/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767231 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767206.923223:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3b80 x1862022114657408/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767231 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767206.923229:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767206.923230:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767206.923232:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114657408:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775767206.923236:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114657536:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775767206.923246:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767206.923251:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767206/real 1775767206]  req@ffff9f14852d0e00 x1862022114657536/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767231 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767206.923258:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0e00 x1862022114657536/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767231 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767206.923263:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767206.923264:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767206.923266:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114657536:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775767206.923269:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114657664:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775767206.923279:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767206.923284:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767206/real 1775767206]  req@ffff9f14852d1880 x1862022114657664/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767231 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767206.923292:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1880 x1862022114657664/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767231 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767206.923297:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767206.923298:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767206.923300:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114657664:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775767206.923303:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114657792:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775767206.923312:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767206.923317:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767206/real 1775767206]  req@ffff9f14852d3100 x1862022114657792/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767231 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767206.923324:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3100 x1862022114657792/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767231 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767206.923329:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767206.923331:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767206.923333:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114657792:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775767206.923347:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114657920:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775767206.923356:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767206.923361:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767206/real 1775767206]  req@ffff9f14852d2300 x1862022114657920/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767231 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767206.923368:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2300 x1862022114657920/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767231 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767206.923378:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767206.923380:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767206.923382:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114657920:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775767207.178729:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775767207.946640:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767208.202743:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775767208.970671:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767209.226753:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775767209.994721:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767210.250763:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775767211.018623:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767211.274743:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1775767212.042647:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:3.0:1775767212.042651:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:2.0:1775767212.042656:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1775767212.042660:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767212.042680:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4589
00000100:00080000:2.0:1775767212.042691:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767212.042697:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775767212.042734:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3480 x1862022114658048/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767212.042763:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767212.042767:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775767212.042769:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767212.042773:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4589
00000100:00080000:2.0:1775767212.042776:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767212.042780:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767212.042788:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775767212.042798:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1c00 x1862022114658176/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767212.042807:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767212.042810:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775767212.042811:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:1.0:1775767212.042816:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114658048:192.168.202.110@tcp:250:kworker.0
00000100:00080000:2.0:1775767212.042817:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4589
00000100:00080000:2.0:1775767212.042821:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767212.042827:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767212.042830:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775767212.042839:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2300 x1862022114658304/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767212.042845:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767212.042848:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775767212.042849:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767212.042851:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4589
00000100:00080000:2.0:1775767212.042854:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767212.042856:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767212.042859:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775767212.042874:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3100 x1862022114658432/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767212.042881:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767212.042884:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000800:00000100:1.0:1775767212.042888:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:2.0:1775767212.042888:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767212.042892:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4589
00000100:00080000:2.0:1775767212.042895:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767212.042898:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767212.042902:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775767212.042910:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1880 x1862022114658560/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767212.042916:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767212.042919:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775767212.042920:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767212.042924:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4589
00000100:00080000:2.0:1775767212.042927:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767212.042931:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767212.042934:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767212.042935:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767212/real 1775767212]  req@ffff9f14852d3480 x1862022114658048/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767237 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767212.042944:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0e00 x1862022114658688/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767212.042952:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:1.0:1775767212.042956:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3480 x1862022114658048/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767237 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767212.042956:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775767212.042958:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767212.042967:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4589
00000100:00080000:2.0:1775767212.042972:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767212.042976:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767212.042976:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767212.042983:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00080000:1.0:1775767212.042985:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767212.042990:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114658048:192.168.202.110@tcp:250:kworker.0
00000100:00100000:2.0:1775767212.042993:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3b80 x1862022114658816/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767212.043000:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767212.043003:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775767212.043005:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767212.043007:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4589
00000100:00080000:2.0:1775767212.043010:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767212.043013:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775767212.043015:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114658176:192.168.202.110@tcp:38:kworker.0
00000008:00100000:2.0:1775767212.043017:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775767212.043028:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0000 x1862022114658944/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:1.0:1775767212.043040:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767212.043052:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767212/real 1775767212]  req@ffff9f14852d1c00 x1862022114658176/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767237 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767212.043061:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1c00 x1862022114658176/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767237 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767212.043066:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767212.043068:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767212.043070:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114658176:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775767212.043075:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114658304:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775767212.043086:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767212.043091:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767212/real 1775767212]  req@ffff9f14852d2300 x1862022114658304/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767237 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767212.043099:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2300 x1862022114658304/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767237 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767212.043104:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767212.043106:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767212.043108:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114658304:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775767212.043112:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114658432:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775767212.043129:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767212.043139:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767212/real 1775767212]  req@ffff9f14852d3100 x1862022114658432/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767237 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767212.043151:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3100 x1862022114658432/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767237 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767212.043159:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767212.043163:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767212.043166:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114658432:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775767212.043173:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114658560:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775767212.043191:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767212.043199:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767212/real 1775767212]  req@ffff9f14852d1880 x1862022114658560/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767237 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767212.043208:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1880 x1862022114658560/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767237 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767212.043213:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767212.043215:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767212.043217:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114658560:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775767212.043221:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114658688:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775767212.043233:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767212.043243:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767212/real 1775767212]  req@ffff9f14852d0e00 x1862022114658688/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767237 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767212.043256:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0e00 x1862022114658688/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767237 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767212.043264:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767212.043268:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767212.043271:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114658688:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775767212.043277:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114658816:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775767212.043295:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767212.043303:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767212/real 1775767212]  req@ffff9f14852d3b80 x1862022114658816/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767237 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767212.043310:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3b80 x1862022114658816/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767237 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767212.043320:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767212.043322:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767212.043324:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114658816:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775767212.043341:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114658944:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775767212.043359:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767212.043369:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767212/real 1775767212]  req@ffff9f14852d0000 x1862022114658944/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767237 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767212.043381:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0000 x1862022114658944/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767237 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767212.043390:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767212.043393:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767212.043396:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114658944:192.168.202.110@tcp:8:kworker.0
00010000:00010000:3.0:1775767212.298746:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775767213.066616:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767213.322708:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775767214.090717:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767214.346789:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775767215.114678:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767215.370685:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775767216.138658:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767216.394712:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767217.162646:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:0.0:1775767217.162653:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767217.162659:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1775767217.162663:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767217.162671:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4594
00000100:00080000:0.0:1775767217.162680:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767217.162685:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775767217.162723:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3b80 x1862022114659072/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767217.162748:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767217.162751:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775767217.162753:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767217.162756:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4594
00000100:00080000:0.0:1775767217.162758:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767217.162760:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767217.162773:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767217.162783:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d1f80 x1862022114659200/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767217.162790:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767217.162793:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:0.0:1775767217.162794:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767217.162797:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4594
00000100:00080000:0.0:1775767217.162798:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767217.162800:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767217.162804:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767217.162808:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114659072:192.168.202.110@tcp:250:kworker.0
00000100:00100000:0.0:1775767217.162813:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0380 x1862022114659328/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767217.162824:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767217.162826:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775767217.162827:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767217.162829:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4594
00000100:00080000:0.0:1775767217.162832:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767217.162834:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767217.162837:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775767217.162846:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3480 x1862022114659456/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767217.162851:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767217.162854:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1775767217.162855:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767217.162857:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4594
00000100:00080000:0.0:1775767217.162860:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767217.162862:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767217.162865:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775767217.162871:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d1180 x1862022114659584/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767217.162877:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767217.162879:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775767217.162880:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767217.162882:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4594
00000100:00080000:0.0:1775767217.162885:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767217.162890:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000800:00000100:1.0:1775767217.162892:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000008:00100000:0.0:1775767217.162894:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767217.162902:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0e00 x1862022114659712/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767217.162913:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767217.162914:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:0.0:1775767217.162915:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767217.162917:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4594
00000100:00080000:0.0:1775767217.162920:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767217.162923:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767217.162927:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767217.162934:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767217/real 1775767217]  req@ffff9f14a04d3b80 x1862022114659072/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767242 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767217.162936:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0a80 x1862022114659840/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767217.162960:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767217.162965:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775767217.162966:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767217.162970:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4594
00000100:00080000:0.0:1775767217.162973:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767217.162978:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775767217.162979:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3b80 x1862022114659072/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767242 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:0.0:1775767217.162982:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775767217.162992:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d2a00 x1862022114659968/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767217.162997:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767217.163005:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767217.163011:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114659072:192.168.202.110@tcp:250:kworker.0
00000100:00100000:1.0:1775767217.163021:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114659200:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775767217.163057:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767217.163073:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767217/real 1775767217]  req@ffff9f14a04d1f80 x1862022114659200/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767242 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767217.163087:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1f80 x1862022114659200/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767242 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767217.163097:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767217.163101:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767217.163104:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114659200:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775767217.163128:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114659328:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775767217.163151:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767217.163163:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767217/real 1775767217]  req@ffff9f14a04d0380 x1862022114659328/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767242 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767217.163176:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0380 x1862022114659328/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767242 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767217.163184:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767217.163188:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767217.163192:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114659328:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775767217.163198:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114659456:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775767217.163221:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767217.163232:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767217/real 1775767217]  req@ffff9f14a04d3480 x1862022114659456/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767242 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767217.163245:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3480 x1862022114659456/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767242 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767217.163264:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767217.163268:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767217.163271:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114659456:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775767217.163277:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114659584:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775767217.163297:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767217.163308:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767217/real 1775767217]  req@ffff9f14a04d1180 x1862022114659584/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767242 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767217.163320:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1180 x1862022114659584/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767242 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767217.163328:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767217.163331:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767217.163334:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114659584:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775767217.163340:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114659712:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775767217.163358:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767217.163369:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767217/real 1775767217]  req@ffff9f14a04d0e00 x1862022114659712/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767242 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767217.163381:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0e00 x1862022114659712/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767242 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767217.163397:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767217.163401:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767217.163404:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114659712:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775767217.163411:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114659840:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775767217.163432:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767217.163444:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767217/real 1775767217]  req@ffff9f14a04d0a80 x1862022114659840/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767242 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767217.163456:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0a80 x1862022114659840/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767242 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767217.163464:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767217.163468:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767217.163472:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114659840:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775767217.163478:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114659968:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775767217.163499:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767217.163509:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767217/real 1775767217]  req@ffff9f14a04d2a00 x1862022114659968/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767242 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767217.163520:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d2a00 x1862022114659968/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767242 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767217.163528:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767217.163531:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767217.163535:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114659968:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775767217.418711:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767218.186684:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767218.442715:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767219.210729:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767219.466708:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767220.234649:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767220.490736:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767221.258634:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767221.514747:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767222.282634:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:2.0:1775767222.282642:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767222.282648:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1775767222.282651:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767222.282659:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4599
00000100:00080000:2.0:1775767222.282667:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767222.282673:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775767222.282703:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0380 x1862022114660096/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767222.282729:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767222.282732:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775767222.282733:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767222.282736:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4599
00000100:00080000:2.0:1775767222.282739:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767222.282741:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767222.282750:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775767222.282759:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1500 x1862022114660224/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767222.282767:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767222.282770:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775767222.282773:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:1.0:1775767222.282774:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114660096:192.168.202.110@tcp:250:kworker.0
00000100:00080000:2.0:1775767222.282781:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4599
00000100:00080000:2.0:1775767222.282784:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767222.282787:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767222.282791:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775767222.282799:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2d80 x1862022114660352/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767222.282806:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767222.282808:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775767222.282809:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767222.282812:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4599
00000100:00080000:2.0:1775767222.282814:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767222.282817:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767222.282820:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775767222.282829:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1180 x1862022114660480/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:1.0:1775767222.282830:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:2.0:1775767222.282838:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767222.282841:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1775767222.282843:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767222.282845:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4599
00000100:00080000:2.0:1775767222.282848:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767222.282850:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767222.282854:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775767222.282860:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2a00 x1862022114660608/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767222.282866:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:1.0:1775767222.282868:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767222/real 1775767222]  req@ffff9f14852d0380 x1862022114660096/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767247 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767222.282870:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775767222.282873:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767222.282881:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4599
00000100:00080000:2.0:1775767222.282885:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767222.282888:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767222.282891:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767222.282892:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0380 x1862022114660096/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767247 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767222.282901:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2680 x1862022114660736/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767222.282903:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767222.282920:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767222.282923:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:2.0:1775767222.282925:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775767222.282928:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:1.0:1775767222.282929:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114660096:192.168.202.110@tcp:250:kworker.0
00000100:00080000:2.0:1775767222.282935:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4599
00000100:00080000:2.0:1775767222.282938:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767222.282941:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767222.282945:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767222.282954:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114660224:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767222.282958:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1f80 x1862022114660864/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767222.282971:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000800:00000100:1.0:1775767222.282976:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:2.0:1775767222.282979:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775767222.282982:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:1.0:1775767222.282988:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767222/real 1775767222]  req@ffff9f14852d1500 x1862022114660224/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767247 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767222.282989:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4599
00000100:00080000:2.0:1775767222.283006:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:1.0:1775767222.283010:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1500 x1862022114660224/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767247 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767222.283012:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767222.283021:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00080000:1.0:1775767222.283022:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767222.283028:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767222.283030:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114660224:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775767222.283035:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114660352:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767222.283041:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0a80 x1862022114660992/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:1.0:1775767222.283050:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767222.283062:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767222/real 1775767222]  req@ffff9f14852d2d80 x1862022114660352/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767247 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767222.283070:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2d80 x1862022114660352/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767247 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767222.283087:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767222.283089:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767222.283092:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114660352:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775767222.283095:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114660480:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775767222.283105:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767222.283110:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767222/real 1775767222]  req@ffff9f14852d1180 x1862022114660480/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767247 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767222.283117:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1180 x1862022114660480/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767247 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767222.283122:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767222.283124:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767222.283126:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114660480:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775767222.283129:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114660608:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775767222.283137:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767222.283142:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767222/real 1775767222]  req@ffff9f14852d2a00 x1862022114660608/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767247 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767222.283149:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2a00 x1862022114660608/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767247 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767222.283154:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767222.283156:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767222.283158:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114660608:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775767222.283162:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114660736:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775767222.283170:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767222.283175:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767222/real 1775767222]  req@ffff9f14852d2680 x1862022114660736/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767247 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767222.283181:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2680 x1862022114660736/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767247 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767222.283186:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767222.283188:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767222.283190:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114660736:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775767222.283202:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114660864:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775767222.283211:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767222.283223:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767222/real 1775767222]  req@ffff9f14852d1f80 x1862022114660864/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767247 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767222.283231:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1f80 x1862022114660864/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767247 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767222.283236:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767222.283238:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767222.283240:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114660864:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775767222.283244:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114660992:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775767222.283254:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767222.283260:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767222/real 1775767222]  req@ffff9f14852d0a80 x1862022114660992/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767247 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767222.283267:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0a80 x1862022114660992/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767247 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767222.283272:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767222.283274:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767222.283275:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114660992:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775767222.538715:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767223.306647:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767223.562728:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767224.330615:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767224.586678:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767225.354618:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767225.610736:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767226.378630:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767226.634703:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767227.402650:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:1.0:1775767227.402671:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767227.402677:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1775767227.402682:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767227.402693:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4604
00000100:00080000:1.0:1775767227.402703:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767227.402710:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775767227.402742:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fd880 x1862022114661120/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767227.402765:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767227.402768:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775767227.402769:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767227.402772:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4604
00000100:00080000:1.0:1775767227.402775:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767227.402779:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767227.402790:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767227.402797:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114661120:192.168.202.110@tcp:250:kworker.0
00000100:00100000:1.0:1775767227.402804:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fc380 x1862022114661248/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767227.402817:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767227.402821:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775767227.402823:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767227.402836:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4604
00000100:00080000:1.0:1775767227.402847:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000800:00000100:0.0:1775767227.402854:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:1.0:1775767227.402854:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767227.402861:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767227.402872:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fe300 x1862022114661376/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767227.402880:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767227.402882:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775767227.402883:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767227.402887:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4604
00000100:00080000:1.0:1775767227.402890:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1775767227.402893:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767227/real 1775767227]  req@ffff9f14ab4fd880 x1862022114661120/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767252 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767227.402895:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767227.402900:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775767227.402910:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff100 x1862022114661504/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767227.402917:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1775767227.402922:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fd880 x1862022114661120/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767252 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767227.402922:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1775767227.402925:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767227.402933:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4604
00000100:00080000:1.0:1775767227.402938:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767227.402943:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00080000:0.0:1775767227.402947:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000008:00100000:1.0:1775767227.402947:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00080000:0.0:1775767227.402955:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767227.402956:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdc00 x1862022114661632/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775767227.402965:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114661120:192.168.202.110@tcp:250:kworker.0
00000100:00080000:1.0:1775767227.402966:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767227.402973:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775767227.402975:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767227.402979:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4604
00000100:00080000:1.0:1775767227.402981:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767227.402985:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767227.402987:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767227.402993:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdf80 x1862022114661760/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775767227.402996:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114661248:192.168.202.110@tcp:38:kworker.0
00000100:00080000:1.0:1775767227.403008:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767227.403010:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775767227.403012:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767227.403014:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4604
00000100:00080000:1.0:1775767227.403021:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767227.403025:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000800:00000100:0.0:1775767227.403027:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000008:00100000:1.0:1775767227.403032:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767227.403041:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fce00 x1862022114661888/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775767227.403043:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767227/real 1775767227]  req@ffff9f14ab4fc380 x1862022114661248/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767252 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767227.403063:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767227.403066:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00100000:0.0:1775767227.403068:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fc380 x1862022114661248/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767252 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767227.403068:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767227.403076:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4604
00000100:00080000:1.0:1775767227.403081:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767227.403083:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767227.403086:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00080000:0.0:1775767227.403090:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767227.403093:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114661248:192.168.202.110@tcp:38:kworker.0
00000008:00100000:1.0:1775767227.403094:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775767227.403105:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114661376:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775767227.403112:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fea00 x1862022114662016/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1775767227.403120:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767227.403132:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767227/real 1775767227]  req@ffff9f14ab4fe300 x1862022114661376/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767252 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767227.403140:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fe300 x1862022114661376/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767252 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767227.403146:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767227.403148:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767227.403150:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114661376:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767227.403154:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114661504:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767227.403164:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767227.403169:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767227/real 1775767227]  req@ffff9f14ab4ff100 x1862022114661504/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767252 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767227.403176:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff100 x1862022114661504/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767252 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767227.403181:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767227.403183:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767227.403185:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114661504:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775767227.403188:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114661632:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767227.403196:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767227.403201:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767227/real 1775767227]  req@ffff9f14ab4fdc00 x1862022114661632/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767252 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767227.403208:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdc00 x1862022114661632/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767252 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767227.403213:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767227.403215:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767227.403216:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114661632:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775767227.403226:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114661760:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767227.403235:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767227.403240:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767227/real 1775767227]  req@ffff9f14ab4fdf80 x1862022114661760/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767252 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767227.403247:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdf80 x1862022114661760/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767252 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767227.403252:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767227.403254:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767227.403256:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114661760:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767227.403259:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114661888:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767227.403267:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767227.403272:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767227/real 1775767227]  req@ffff9f14ab4fce00 x1862022114661888/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767252 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767227.403279:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fce00 x1862022114661888/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767252 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767227.403288:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767227.403290:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767227.403292:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114661888:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767227.403295:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114662016:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767227.403303:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767227.403308:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767227/real 1775767227]  req@ffff9f14ab4fea00 x1862022114662016/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767252 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767227.403314:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fea00 x1862022114662016/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767252 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767227.403319:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767227.403321:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767227.403323:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114662016:192.168.202.110@tcp:8:kworker.0
00010000:00010000:3.0:1775767227.658808:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767228.426654:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767228.682707:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767229.450618:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775767229.706688:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767230.474666:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767230.730669:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767231.498660:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767231.754742:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1775767232.522643:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767232.522654:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1775767232.522659:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00100000:2.0:1775767232.522666:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:1.0:1775767232.522669:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4610
00000100:00080000:1.0:1775767232.522687:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767232.522696:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775767232.522740:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff800 x1862022114662144/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767232.522771:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767232.522774:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775767232.522776:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767232.522780:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4610
00000100:00080000:1.0:1775767232.522783:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767232.522786:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767232.522796:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767232.522806:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fed80 x1862022114662272/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767232.522814:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767232.522817:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775767232.522818:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767232.522821:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4610
00000100:00080000:1.0:1775767232.522823:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767232.522826:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767232.522829:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767232.522836:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114662144:192.168.202.110@tcp:250:kworker.0
00000100:00100000:1.0:1775767232.522840:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ffb80 x1862022114662400/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767232.522852:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767232.522855:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775767232.522856:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767232.522859:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4610
00000100:00080000:1.0:1775767232.522861:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767232.522864:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767232.522867:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775767232.522875:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fca80 x1862022114662528/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767232.522886:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767232.522889:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1775767232.522891:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767232.522896:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4610
00000100:00080000:1.0:1775767232.522913:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767232.522918:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767232.522924:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:0.0:1775767232.522925:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767232.522943:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fea00 x1862022114662656/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767232.522954:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767232.522958:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775767232.522960:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767232.522966:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4610
00000100:00100000:0.0:1775767232.522971:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767232/real 1775767232]  req@ffff9f14ab4ff800 x1862022114662144/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767257 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767232.522971:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767232.522981:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767232.522989:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767232.522994:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff800 x1862022114662144/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767257 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767232.523004:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00100000:1.0:1775767232.523006:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fce00 x1862022114662784/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767232.523012:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767232.523020:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114662144:192.168.202.110@tcp:250:kworker.0
00000100:00080000:1.0:1775767232.523022:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767232.523025:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775767232.523027:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775767232.523029:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114662272:192.168.202.110@tcp:38:kworker.0
00000100:00080000:1.0:1775767232.523033:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4610
00000100:00080000:1.0:1775767232.523038:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767232.523041:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767232.523046:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000800:00000100:0.0:1775767232.523055:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767232.523059:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdf80 x1862022114662912/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767232.523066:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1775767232.523068:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767232/real 1775767232]  req@ffff9f14ab4fed80 x1862022114662272/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767257 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767232.523071:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775767232.523075:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767232.523084:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4610
00000100:00080000:1.0:1775767232.523089:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1775767232.523092:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fed80 x1862022114662272/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767257 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767232.523093:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767232.523100:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00080000:0.0:1775767232.523109:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00100000:1.0:1775767232.523114:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdc00 x1862022114663040/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767232.523115:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767232.523122:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114662272:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767232.523152:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ffb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114662400:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767232.523167:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767232.523176:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767232/real 1775767232]  req@ffff9f14ab4ffb80 x1862022114662400/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767257 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767232.523185:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ffb80 x1862022114662400/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767257 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767232.523192:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767232.523194:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767232.523198:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ffb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114662400:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767232.523202:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114662528:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767232.523215:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767232.523223:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767232/real 1775767232]  req@ffff9f14ab4fca80 x1862022114662528/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767257 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767232.523230:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fca80 x1862022114662528/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767257 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767232.523236:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767232.523239:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767232.523242:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114662528:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775767232.523246:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114662656:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767232.523269:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767232.523277:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767232/real 1775767232]  req@ffff9f14ab4fea00 x1862022114662656/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767257 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767232.523285:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fea00 x1862022114662656/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767257 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767232.523291:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767232.523293:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767232.523296:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114662656:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775767232.523301:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114662784:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767232.523317:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767232.523325:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767232/real 1775767232]  req@ffff9f14ab4fce00 x1862022114662784/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767257 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767232.523339:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fce00 x1862022114662784/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767257 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767232.523346:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767232.523348:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767232.523350:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114662784:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767232.523354:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114662912:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767232.523368:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767232.523381:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767232/real 1775767232]  req@ffff9f14ab4fdf80 x1862022114662912/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767257 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767232.523394:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdf80 x1862022114662912/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767257 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767232.523400:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767232.523402:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767232.523405:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114662912:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767232.523409:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114663040:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767232.523422:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767232.523429:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767232/real 1775767232]  req@ffff9f14ab4fdc00 x1862022114663040/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767257 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767232.523436:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdc00 x1862022114663040/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767257 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767232.523446:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767232.523449:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767232.523451:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114663040:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775767232.778726:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767233.546703:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767233.802725:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767234.570636:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767234.826799:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767235.594709:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767235.850767:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767236.618726:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775767236.874760:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767237.642690:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775767237.642696:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767237.642707:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775767237.642712:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767237.642724:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4615
00000100:00080000:3.0:1775767237.642736:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767237.642744:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775767237.642784:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747100 x1862022114663168/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767237.642813:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767237.642817:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775767237.642819:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767237.642824:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4615
00000100:00080000:3.0:1775767237.642828:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767237.642833:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767237.642842:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767237.642855:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745c00 x1862022114663296/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767237.642866:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767237.642870:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775767237.642872:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767237.642878:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4615
00000100:00080000:3.0:1775767237.642882:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1775767237.642886:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114663168:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775767237.642890:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767237.642897:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767237.642910:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745180 x1862022114663424/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767237.642927:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767237.642930:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775767237.642933:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767237.642937:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4615
00000100:00080000:3.0:1775767237.642943:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000800:00000100:0.0:1775767237.642949:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775767237.642950:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767237.642959:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775767237.642971:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746680 x1862022114663552/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767237.642982:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767237.642986:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00100000:0.0:1775767237.642990:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767237/real 1775767237]  req@ffff9f1485747100 x1862022114663168/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767262 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767237.642990:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767237.643000:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4615
00000100:00080000:3.0:1775767237.643007:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1775767237.643011:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747100 x1862022114663168/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767262 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767237.643014:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767237.643025:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00080000:0.0:1775767237.643026:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767237.643032:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767237.643035:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114663168:192.168.202.110@tcp:250:kworker.0
00000100:00100000:0.0:1775767237.643043:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114663296:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767237.643043:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744000 x1862022114663680/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1775767237.643064:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775767237.643070:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1775767237.643074:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767237/real 1775767237]  req@ffff9f1485745c00 x1862022114663296/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767262 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767237.643075:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775767237.643080:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767237.643089:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4615
00000100:00100000:0.0:1775767237.643090:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745c00 x1862022114663296/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767262 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767237.643096:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767237.643100:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767237.643105:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00080000:3.0:1775767237.643105:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775767237.643111:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114663296:192.168.202.110@tcp:38:kworker.0
00000008:00100000:3.0:1775767237.643115:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767237.643130:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747800 x1862022114663808/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775767237.643134:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114663424:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775767237.643151:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000800:00000100:0.0:1775767237.643156:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775767237.643157:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775767237.643160:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767237.643166:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4615
00000100:00100000:0.0:1775767237.643167:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767237/real 1775767237]  req@ffff9f1485745180 x1862022114663424/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767262 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767237.643172:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767237.643177:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767237.643185:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767237.643190:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745180 x1862022114663424/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767262 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767237.643197:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767237.643201:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767237.643201:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745f80 x1862022114663936/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775767237.643207:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114663424:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767237.643217:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114663552:192.168.202.110@tcp:8:kworker.0
00000100:00080000:3.0:1775767237.643224:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767237.643229:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775767237.643232:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000800:00000100:0.0:1775767237.643236:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775767237.643241:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4615
00000100:00100000:0.0:1775767237.643245:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767237/real 1775767237]  req@ffff9f1485746680 x1862022114663552/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767262 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767237.643246:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767237.643256:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775767237.643260:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746680 x1862022114663552/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767262 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:3.0:1775767237.643263:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00080000:0.0:1775767237.643269:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767237.643273:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767237.643275:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114663552:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775767237.643282:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114663680:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775767237.643282:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745500 x1862022114664064/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1775767237.643296:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767237.643303:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767237/real 1775767237]  req@ffff9f1485744000 x1862022114663680/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767262 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767237.643314:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744000 x1862022114663680/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767262 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767237.643319:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767237.643321:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767237.643323:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114663680:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775767237.643335:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114663808:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767237.643345:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767237.643351:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767237/real 1775767237]  req@ffff9f1485747800 x1862022114663808/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767262 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767237.643357:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747800 x1862022114663808/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767262 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767237.643362:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767237.643364:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767237.643366:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114663808:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767237.643369:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114663936:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767237.643378:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767237.643383:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767237/real 1775767237]  req@ffff9f1485745f80 x1862022114663936/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767262 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767237.643389:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745f80 x1862022114663936/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767262 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767237.643399:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767237.643400:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767237.643402:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114663936:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767237.643406:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114664064:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767237.643414:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767237.643419:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767237/real 1775767237]  req@ffff9f1485745500 x1862022114664064/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767262 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767237.643441:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745500 x1862022114664064/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767262 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767237.643450:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767237.643453:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767237.643456:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114664064:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775767237.898728:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767238.666733:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767238.922724:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767239.690731:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767239.946760:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767240.714658:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767240.970734:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767241.738625:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767241.994712:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1775767242.762657:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767242.762666:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775767242.762669:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767242.762679:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4620
00000100:00100000:2.0:1775767242.762688:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775767242.762691:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767242.762703:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775767242.762744:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744700 x1862022114664192/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767242.762774:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767242.762778:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775767242.762780:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767242.762784:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4620
00000100:00080000:3.0:1775767242.762788:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767242.762792:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767242.762803:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767242.762820:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745500 x1862022114664320/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775767242.762823:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114664192:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775767242.762840:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767242.762844:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775767242.762845:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767242.762849:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4620
00000100:00080000:3.0:1775767242.762853:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767242.762857:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767242.762861:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767242.762871:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745f80 x1862022114664448/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767242.762880:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000800:00000100:0.0:1775767242.762882:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775767242.762883:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775767242.762885:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767242.762888:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4620
00000100:00080000:3.0:1775767242.762891:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767242.762895:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767242.762900:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775767242.762909:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747800 x1862022114664576/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767242.762917:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767242.762920:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775767242.762924:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767242.762927:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4620
00000100:00080000:3.0:1775767242.762930:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1775767242.762932:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767242/real 1775767242]  req@ffff9f1485744700 x1862022114664192/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767267 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767242.762933:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767242.762938:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775767242.762946:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744000 x1862022114664704/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775767242.762959:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744700 x1862022114664192/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767267 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767242.762962:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767242.762966:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775767242.762967:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767242.762969:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767242.762971:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4620
00000100:00080000:0.0:1775767242.762973:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:3.0:1775767242.762974:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1775767242.762977:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114664192:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775767242.762978:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767242.762983:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767242.762992:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746680 x1862022114664832/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775767242.762998:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114664320:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775767242.763009:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767242.763012:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775767242.763014:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767242.763018:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4620
00000100:00080000:3.0:1775767242.763021:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000800:00000100:0.0:1775767242.763025:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775767242.763025:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767242.763030:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767242.763038:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767242/real 1775767242]  req@ffff9f1485745500 x1862022114664320/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767267 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767242.763040:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745180 x1862022114664960/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767242.763050:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1775767242.763051:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745500 x1862022114664320/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767267 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767242.763054:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775767242.763055:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767242.763059:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4620
00000100:00080000:0.0:1775767242.763060:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767242.763063:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767242.763064:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767242.763067:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114664320:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775767242.763067:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767242.763072:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775767242.763073:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114664448:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767242.763080:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745c00 x1862022114665088/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1775767242.763090:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767242.763100:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767242/real 1775767242]  req@ffff9f1485745f80 x1862022114664448/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767267 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767242.763112:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745f80 x1862022114664448/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767267 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767242.763120:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767242.763123:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767242.763126:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114664448:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767242.763133:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114664576:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767242.763153:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767242.763162:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767242/real 1775767242]  req@ffff9f1485747800 x1862022114664576/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767267 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767242.763179:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747800 x1862022114664576/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767267 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767242.763184:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767242.763186:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767242.763188:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114664576:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775767242.763192:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114664704:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767242.763204:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767242.763210:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767242/real 1775767242]  req@ffff9f1485744000 x1862022114664704/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767267 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767242.763217:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744000 x1862022114664704/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767267 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767242.763222:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767242.763224:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767242.763226:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114664704:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775767242.763237:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114664832:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767242.763248:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767242.763253:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767242/real 1775767242]  req@ffff9f1485746680 x1862022114664832/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767267 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767242.763260:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746680 x1862022114664832/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767267 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767242.763270:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767242.763272:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767242.763273:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114664832:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767242.763276:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114664960:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767242.763286:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767242.763291:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767242/real 1775767242]  req@ffff9f1485745180 x1862022114664960/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767267 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767242.763297:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745180 x1862022114664960/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767267 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767242.763302:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767242.763303:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767242.763305:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114664960:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767242.763308:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114665088:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767242.763317:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767242.763321:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767242/real 1775767242]  req@ffff9f1485745c00 x1862022114665088/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767267 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767242.763328:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745c00 x1862022114665088/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767267 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767242.763332:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767242.763334:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767242.763337:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114665088:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775767243.018753:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767243.786652:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767244.042731:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767244.810664:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767245.066741:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767245.834641:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767246.090728:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767246.858668:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767247.114767:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767247.882648:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775767247.882680:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767247.882688:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775767247.882693:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767247.882701:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4625
00000100:00080000:3.0:1775767247.882710:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767247.882715:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775767247.882749:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747100 x1862022114665216/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767247.882773:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767247.882776:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775767247.882777:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767247.882780:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4625
00000100:00080000:3.0:1775767247.882783:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767247.882785:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767247.882792:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767247.882800:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744e00 x1862022114665344/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767247.882808:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767247.882812:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775767247.882814:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767247.882817:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4625
00000100:00080000:3.0:1775767247.882819:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767247.882821:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767247.882824:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767247.882830:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744380 x1862022114665472/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767247.882836:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767247.882837:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775767247.882839:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767247.882841:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4625
00000100:00080000:3.0:1775767247.882844:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1775767247.882847:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114665216:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775767247.882848:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767247.882855:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775767247.882861:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746a00 x1862022114665600/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767247.882867:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767247.882869:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775767247.882870:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767247.882873:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4625
00000100:00080000:3.0:1775767247.882875:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767247.882877:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767247.882881:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775767247.882888:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744a80 x1862022114665728/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767247.882895:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767247.882897:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775767247.882898:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767247.882900:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4625
00000100:00080000:3.0:1775767247.882902:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767247.882906:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767247.882909:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767247.882918:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747480 x1862022114665856/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1775767247.882919:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775767247.882928:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767247.882931:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775767247.882932:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767247.882935:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4625
00000100:00080000:3.0:1775767247.882938:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767247.882940:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767247.882943:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767247.882951:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745880 x1862022114665984/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767247.882957:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1775767247.882960:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767247/real 1775767247]  req@ffff9f1485747100 x1862022114665216/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767272 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767247.882961:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775767247.882965:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767247.882973:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4625
00000100:00080000:3.0:1775767247.882977:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767247.882979:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767247.882982:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775767247.882984:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747100 x1862022114665216/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767272 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767247.882992:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746300 x1862022114666112/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767247.882993:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767247.883001:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767247.883007:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114665216:192.168.202.110@tcp:250:kworker.0
00000100:00100000:0.0:1775767247.883013:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114665344:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767247.883030:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767247.883038:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767247/real 1775767247]  req@ffff9f1485744e00 x1862022114665344/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767272 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767247.883046:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744e00 x1862022114665344/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767272 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767247.883051:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767247.883053:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767247.883056:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114665344:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767247.883073:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114665472:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767247.883100:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767247.883107:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767247/real 1775767247]  req@ffff9f1485744380 x1862022114665472/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767272 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767247.883115:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744380 x1862022114665472/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767272 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767247.883121:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767247.883122:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767247.883125:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114665472:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767247.883129:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114665600:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767247.883140:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767247.883145:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767247/real 1775767247]  req@ffff9f1485746a00 x1862022114665600/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767272 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767247.883153:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746a00 x1862022114665600/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767272 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767247.883159:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767247.883160:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767247.883162:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114665600:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775767247.883166:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114665728:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767247.883175:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767247.883180:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767247/real 1775767247]  req@ffff9f1485744a80 x1862022114665728/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767272 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767247.883188:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744a80 x1862022114665728/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767272 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767247.883194:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767247.883196:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767247.883198:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114665728:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775767247.883201:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114665856:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767247.883211:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767247.883216:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767247/real 1775767247]  req@ffff9f1485747480 x1862022114665856/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767272 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767247.883223:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747480 x1862022114665856/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767272 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767247.883228:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767247.883229:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767247.883231:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114665856:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767247.883234:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114665984:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767247.883244:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767247.883249:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767247/real 1775767247]  req@ffff9f1485745880 x1862022114665984/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767272 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767247.883257:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745880 x1862022114665984/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767272 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767247.883261:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767247.883263:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767247.883266:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114665984:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767247.883269:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114666112:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767247.883279:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767247.883284:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767247/real 1775767247]  req@ffff9f1485746300 x1862022114666112/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767272 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767247.883291:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746300 x1862022114666112/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767272 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767247.883313:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767247.883315:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767247.883317:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114666112:192.168.202.110@tcp:8:kworker.0
00010000:00010000:3.0:1775767248.138737:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767248.906652:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767249.162722:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767249.930647:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767250.186750:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767250.954658:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767251.210737:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767251.978677:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767252.234750:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767253.002691:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:0.0:1775767253.002708:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767253.002717:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1775767253.002720:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767253.002729:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4630
00000100:00080000:0.0:1775767253.002739:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767253.002745:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775767253.002778:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3800 x1862022114666240/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767253.002804:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767253.002807:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775767253.002808:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767253.002811:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4630
00000100:00080000:0.0:1775767253.002813:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767253.002817:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767253.002825:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767253.002834:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d1500 x1862022114666368/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767253.002842:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767253.002845:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:0.0:1775767253.002846:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767253.002851:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4630
00000100:00100000:1.0:1775767253.002853:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114666240:192.168.202.110@tcp:250:kworker.0
00000100:00080000:0.0:1775767253.002854:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767253.002858:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767253.002864:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767253.002871:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0000 x1862022114666496/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767253.002876:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767253.002878:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775767253.002879:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767253.002882:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4630
00000100:00080000:0.0:1775767253.002884:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767253.002886:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767253.002889:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775767253.002897:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d2300 x1862022114666624/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767253.002902:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767253.002905:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1775767253.002906:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767253.002908:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4630
00000100:00080000:0.0:1775767253.002917:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767253.002920:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767253.002923:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:1.0:1775767253.002925:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767253.002942:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3100 x1862022114666752/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767253.002949:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767253.002952:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775767253.002953:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767253.002956:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4630
00000100:00080000:0.0:1775767253.002960:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:1.0:1775767253.002963:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767253/real 1775767253]  req@ffff9f14a04d3800 x1862022114666240/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767278 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767253.002964:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767253.002969:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767253.002977:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d2a00 x1862022114666880/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767253.003000:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767253.003004:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00100000:1.0:1775767253.003007:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3800 x1862022114666240/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767278 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767253.003008:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767253.003018:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4630
00000100:00080000:1.0:1775767253.003022:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767253.003023:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767253.003028:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:0.0:1775767253.003032:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775767253.003035:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114666240:192.168.202.110@tcp:250:kworker.0
00000008:00100000:0.0:1775767253.003043:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767253.003057:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114666368:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767253.003060:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0a80 x1862022114667008/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767253.003074:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767253.003077:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000800:00000100:1.0:1775767253.003080:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:0.0:1775767253.003081:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767253.003087:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4630
00000100:00080000:0.0:1775767253.003092:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:1.0:1775767253.003095:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767253/real 1775767253]  req@ffff9f14a04d1500 x1862022114666368/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767278 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767253.003097:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767253.003104:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775767253.003123:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1500 x1862022114666368/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767278 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767253.003127:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0e00 x1862022114667136/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767253.003147:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767253.003152:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767253.003156:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114666368:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775767253.003161:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114666496:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775767253.003175:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767253.003181:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767253/real 1775767253]  req@ffff9f14a04d0000 x1862022114666496/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767278 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767253.003189:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0000 x1862022114666496/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767278 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767253.003194:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767253.003196:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767253.003197:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114666496:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775767253.003201:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114666624:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775767253.003211:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767253.003217:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767253/real 1775767253]  req@ffff9f14a04d2300 x1862022114666624/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767278 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767253.003224:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d2300 x1862022114666624/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767278 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767253.003229:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767253.003231:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767253.003233:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114666624:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775767253.003236:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114666752:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775767253.003253:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767253.003259:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767253/real 1775767253]  req@ffff9f14a04d3100 x1862022114666752/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767278 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767253.003266:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3100 x1862022114666752/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767278 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767253.003272:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767253.003274:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767253.003277:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114666752:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775767253.003282:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114666880:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775767253.003323:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767253.003334:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767253/real 1775767253]  req@ffff9f14a04d2a00 x1862022114666880/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767278 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767253.003347:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d2a00 x1862022114666880/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767278 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767253.003356:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767253.003359:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767253.003362:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114666880:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775767253.003377:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114667008:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775767253.003392:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767253.003398:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767253/real 1775767253]  req@ffff9f14a04d0a80 x1862022114667008/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767278 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767253.003406:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0a80 x1862022114667008/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767278 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767253.003411:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767253.003413:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767253.003415:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114667008:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775767253.003418:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114667136:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775767253.003427:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767253.003432:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767253/real 1775767253]  req@ffff9f14a04d0e00 x1862022114667136/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767278 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767253.003438:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0e00 x1862022114667136/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767278 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767253.003443:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767253.003445:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767253.003447:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114667136:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775767253.258767:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767254.026707:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767254.282726:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767255.050744:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767255.306704:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767256.074686:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767256.330742:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767257.098628:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767257.354754:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767258.122716:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775767258.122720:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767258.122729:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775767258.122734:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767258.122745:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4635
00000100:00080000:3.0:1775767258.122758:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767258.122766:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775767258.122812:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747b80 x1862022114667264/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767258.122841:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767258.122844:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775767258.122846:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767258.122850:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4635
00000100:00080000:3.0:1775767258.122853:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767258.122856:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767258.122867:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767258.122880:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746300 x1862022114667392/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775767258.122886:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114667264:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775767258.122896:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767258.122899:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775767258.122900:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767258.122903:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4635
00000100:00080000:3.0:1775767258.122905:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767258.122908:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767258.122913:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767258.122922:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745880 x1862022114667520/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767258.122943:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767258.122945:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775767258.122947:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767258.122950:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4635
00000100:00080000:3.0:1775767258.122953:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767258.122956:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767258.122960:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775767258.122967:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747480 x1862022114667648/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767258.122972:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767258.122974:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775767258.122976:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767258.122979:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4635
00000100:00080000:3.0:1775767258.122982:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767258.122984:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767258.122987:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775767258.122997:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744a80 x1862022114667776/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1775767258.122998:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775767258.123006:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767258.123009:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775767258.123011:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767258.123016:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4635
00000100:00080000:3.0:1775767258.123020:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767258.123025:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767258.123031:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767258.123060:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746a00 x1862022114667904/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767258.123070:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1775767258.123073:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767258/real 1775767258]  req@ffff9f1485747b80 x1862022114667264/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767283 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767258.123075:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775767258.123079:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767258.123088:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4635
00000100:00080000:3.0:1775767258.123095:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767258.123099:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775767258.123104:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747b80 x1862022114667264/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767283 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:3.0:1775767258.123104:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767258.123116:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744380 x1862022114668032/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767258.123135:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767258.123135:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767258.123142:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775767258.123144:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767258.123145:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:3.0:1775767258.123150:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4635
00000100:00100000:0.0:1775767258.123153:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114667264:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775767258.123155:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767258.123162:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767258.123166:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775767258.123173:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744e00 x1862022114668160/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775767258.123184:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114667392:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767258.123211:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767258.123221:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767258/real 1775767258]  req@ffff9f1485746300 x1862022114667392/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767283 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767258.123230:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746300 x1862022114667392/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767283 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767258.123236:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767258.123256:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767258.123259:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114667392:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767258.123264:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114667520:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767258.123281:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767258.123289:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767258/real 1775767258]  req@ffff9f1485745880 x1862022114667520/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767283 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767258.123297:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745880 x1862022114667520/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767283 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767258.123303:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767258.123305:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767258.123307:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114667520:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767258.123311:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114667648:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767258.123323:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767258.123329:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767258/real 1775767258]  req@ffff9f1485747480 x1862022114667648/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767283 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767258.123337:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747480 x1862022114667648/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767283 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767258.123342:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767258.123344:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767258.123345:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114667648:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775767258.123349:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114667776:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767258.123359:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767258.123364:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767258/real 1775767258]  req@ffff9f1485744a80 x1862022114667776/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767283 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767258.123372:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744a80 x1862022114667776/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767283 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767258.123377:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767258.123378:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767258.123380:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114667776:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775767258.123384:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114667904:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767258.123393:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767258.123399:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767258/real 1775767258]  req@ffff9f1485746a00 x1862022114667904/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767283 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767258.123410:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746a00 x1862022114667904/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767283 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767258.123415:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767258.123417:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767258.123419:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114667904:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767258.123422:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114668032:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767258.123432:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767258.123444:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767258/real 1775767258]  req@ffff9f1485744380 x1862022114668032/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767283 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767258.123451:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744380 x1862022114668032/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767283 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767258.123456:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767258.123459:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767258.123461:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114668032:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767258.123464:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114668160:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767258.123475:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767258.123481:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767258/real 1775767258]  req@ffff9f1485744e00 x1862022114668160/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767283 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767258.123488:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744e00 x1862022114668160/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767283 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767258.123498:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767258.123500:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767258.123502:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114668160:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775767258.378697:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767259.146676:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767259.402775:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767260.170733:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767260.426771:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767261.194692:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767261.450704:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767262.218714:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767262.474787:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1775767263.242598:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767263.242607:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775767263.242611:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767263.242623:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4640
00000100:00080000:3.0:1775767263.242635:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767263.242643:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775767263.242654:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:3.0:1775767263.242684:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747100 x1862022114668288/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767263.242707:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767263.242710:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775767263.242712:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767263.242715:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4640
00000100:00080000:3.0:1775767263.242719:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767263.242723:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767263.242732:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767263.242743:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114668288:192.168.202.110@tcp:250:kworker.0
00000100:00100000:3.0:1775767263.242754:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745c00 x1862022114668416/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767263.242763:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767263.242766:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775767263.242768:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767263.242771:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4640
00000100:00080000:3.0:1775767263.242774:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767263.242776:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767263.242779:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767263.242787:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745180 x1862022114668544/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767263.242794:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767263.242796:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775767263.242797:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767263.242799:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4640
00000100:00080000:3.0:1775767263.242802:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767263.242806:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767263.242812:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775767263.242824:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746680 x1862022114668672/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775767263.242835:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114668416:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775767263.242852:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767263.242856:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775767263.242858:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775767263.242862:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114668544:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775767263.242867:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4640
00000100:00080000:3.0:1775767263.242874:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767263.242878:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767263.242884:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775767263.242894:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114668672:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775767263.242898:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744000 x1862022114668800/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767263.242908:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767263.242911:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775767263.242913:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767263.242915:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4640
00000100:00080000:3.0:1775767263.242918:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767263.242921:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767263.242925:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767263.242931:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114668800:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775767263.242937:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747800 x1862022114668928/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767263.242945:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767263.242947:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775767263.242949:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767263.242952:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4640
00000100:00080000:3.0:1775767263.242955:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767263.242958:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767263.242962:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767263.242966:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114668928:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767263.242972:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745f80 x1862022114669056/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767263.242985:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1775767263.242991:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114669056:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775767263.242992:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775767263.242997:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767263.243003:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4640
00000100:00080000:3.0:1775767263.243009:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767263.243014:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767263.243021:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775767263.243035:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745500 x1862022114669184/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775767263.243078:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114669184:192.168.202.110@tcp:8:kworker.0
00010000:00010000:3.0:1775767263.498699:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767264.266767:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767264.522728:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767265.290679:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767265.546708:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767266.314692:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000400:00000100:1.0:1775767266.314831:0:2390:0:(lib-socket.c:454:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.202.110:988
00000400:00000100:1.0:1775767266.314890:0:2390:0:(acceptor.c:111:lnet_connect_console_error()) Connection to 192.168.202.110@tcp at host 192.168.202.110 was unreachable: the network or that node may be down, or Lustre may be misconfigured.
00000800:00000100:1.0:1775767266.314907:0:2390:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:1.0:1775767266.314952:0:2390:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:1.0:1775767266.314961:0:2390:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:1.0:1775767266.314968:0:2390:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:1.0:1775767266.314977:0:2390:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:1.0:1775767266.314984:0:2390:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:1.0:1775767266.314992:0:2390:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:1.0:1775767266.315000:0:2390:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000100:00100000:0.0:1775767266.315054:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767263/real 1775767266]  req@ffff9f1485745500 x1862022114669184/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767288 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767266.315103:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745500 x1862022114669184/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767288 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767266.315127:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767266.315134:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767266.315139:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114669184:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775767266.315149:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767263/real 1775767266]  req@ffff9f1485745f80 x1862022114669056/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767288 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767266.315318:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745f80 x1862022114669056/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767288 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767266.315326:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767266.315330:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767266.315333:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114669056:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767266.315341:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767263/real 1775767266]  req@ffff9f1485747800 x1862022114668928/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767288 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767266.315355:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747800 x1862022114668928/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767288 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767266.315363:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767266.315367:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767266.315371:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114668928:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767266.315379:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767263/real 1775767266]  req@ffff9f1485744000 x1862022114668800/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767288 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767266.315392:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744000 x1862022114668800/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767288 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767266.315400:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767266.315404:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767266.315407:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114668800:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775767266.315415:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767263/real 1775767266]  req@ffff9f1485746680 x1862022114668672/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767288 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767266.315425:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746680 x1862022114668672/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767288 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767266.315432:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767266.315436:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767266.315439:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114668672:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775767266.315445:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767263/real 1775767266]  req@ffff9f1485745c00 x1862022114668416/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767288 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767266.315457:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745c00 x1862022114668416/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767288 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767266.315464:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767266.315468:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767266.315471:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114668416:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767266.315478:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767263/real 1775767266]  req@ffff9f1485745180 x1862022114668544/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767288 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767266.315487:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745180 x1862022114668544/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767288 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767266.315494:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767266.315497:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767266.315500:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114668544:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767266.315507:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767263/real 1775767266]  req@ffff9f1485747100 x1862022114668288/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767288 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767266.315519:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747100 x1862022114668288/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767288 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767266.315526:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767266.315531:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767266.315534:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114668288:192.168.202.110@tcp:250:kworker.0
00010000:00010000:2.0:1775767266.570787:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767267.338712:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767267.594792:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1775767268.362664:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767268.362671:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775767268.362675:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767268.362684:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4645
00000100:00080000:3.0:1775767268.362693:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767268.362699:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775767268.362710:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:3.0:1775767268.362730:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744700 x1862022114669312/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767268.362762:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767268.362766:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775767268.362767:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767268.362770:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4645
00000100:00080000:3.0:1775767268.362773:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767268.362775:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767268.362783:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767268.362794:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747100 x1862022114669440/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767268.362804:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767268.362808:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775767268.362810:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767268.362816:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4645
00000100:00100000:0.0:1775767268.362817:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114669312:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775767268.362820:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767268.362835:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767268.362842:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767268.362859:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745180 x1862022114669568/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767268.362869:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767268.362873:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775767268.362875:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767268.362879:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4645
00000100:00080000:3.0:1775767268.362883:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000800:00000100:0.0:1775767268.362889:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775767268.362890:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767268.362898:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775767268.362912:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745c00 x1862022114669696/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767268.362922:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767268.362932:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775767268.362934:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775767268.362941:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767268/real 1775767268]  req@ffff9f1485744700 x1862022114669312/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767293 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767268.362945:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4645
00000100:00080000:3.0:1775767268.362961:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767268.362966:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775767268.362970:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744700 x1862022114669312/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767293 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:3.0:1775767268.362973:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00080000:0.0:1775767268.362988:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00100000:3.0:1775767268.362991:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746680 x1862022114669824/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767268.362996:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767268.363006:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114669312:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775767268.363016:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1775767268.363017:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114669440:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775767268.363025:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775767268.363029:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767268.363034:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4645
00000100:00080000:3.0:1775767268.363039:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767268.363045:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767268.363052:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000800:00000100:0.0:1775767268.363056:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767268.363072:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767268/real 1775767268]  req@ffff9f1485747100 x1862022114669440/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767293 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767268.363077:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744000 x1862022114669952/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775767268.363094:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747100 x1862022114669440/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767293 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767268.363096:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767268.363104:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775767268.363109:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767268.363117:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767268.363119:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4645
00000100:00080000:0.0:1775767268.363125:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00080000:3.0:1775767268.363126:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1775767268.363133:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114669440:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775767268.363134:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767268.363181:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767268.363197:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747800 x1862022114670080/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767268.363207:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767268.363211:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775767268.363213:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767268.363218:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4645
00000100:00100000:0.0:1775767268.363221:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114669568:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775767268.363223:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767268.363232:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767268.363238:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775767268.363252:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745f80 x1862022114670208/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1775767268.363257:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767268.363274:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767268/real 1775767268]  req@ffff9f1485745180 x1862022114669568/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767293 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767268.363287:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745180 x1862022114669568/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767293 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767268.363302:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767268.363306:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767268.363310:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114669568:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767268.363317:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114669696:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767268.363342:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767268.363361:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767268/real 1775767268]  req@ffff9f1485745c00 x1862022114669696/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767293 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767268.363374:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745c00 x1862022114669696/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767293 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767268.363382:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767268.363385:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767268.363389:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114669696:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775767268.363395:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114669824:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767268.363417:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767268.363430:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767268/real 1775767268]  req@ffff9f1485746680 x1862022114669824/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767293 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767268.363443:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746680 x1862022114669824/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767293 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767268.363451:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767268.363455:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767268.363461:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114669824:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775767268.363468:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114669952:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767268.363491:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767268.363503:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767268/real 1775767268]  req@ffff9f1485744000 x1862022114669952/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767293 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767268.363516:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744000 x1862022114669952/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767293 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767268.363526:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767268.363530:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767268.363534:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114669952:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767268.363542:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114670080:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767268.363609:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767268.363625:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767268/real 1775767268]  req@ffff9f1485747800 x1862022114670080/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767293 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767268.363639:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747800 x1862022114670080/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767293 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767268.363648:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767268.363652:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767268.363655:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114670080:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767268.363677:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114670208:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767268.363702:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767268.363721:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767268/real 1775767268]  req@ffff9f1485745f80 x1862022114670208/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767293 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767268.363736:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745f80 x1862022114670208/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767293 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767268.363745:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767268.363749:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767268.363753:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114670208:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775767268.618748:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767269.386693:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767269.642686:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767270.410662:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767270.666772:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767271.434668:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767271.690763:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767272.458635:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767272.714716:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767273.482671:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775767273.482680:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767273.482688:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775767273.482694:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767273.482704:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4650
00000100:00080000:3.0:1775767273.482717:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767273.482724:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775767273.482770:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745500 x1862022114670336/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767273.482807:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767273.482811:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775767273.482814:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767273.482819:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4650
00000100:00080000:3.0:1775767273.482823:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767273.482834:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767273.482846:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767273.482861:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744e00 x1862022114670464/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767273.482871:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767273.482875:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775767273.482877:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767273.482882:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4650
00000100:00080000:3.0:1775767273.482887:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1775767273.482890:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114670336:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775767273.482894:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767273.482904:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767273.482915:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744380 x1862022114670592/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767273.482926:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767273.482929:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775767273.482932:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767273.482936:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4650
00000100:00080000:3.0:1775767273.482940:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767273.482945:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767273.482950:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775767273.482961:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746a00 x1862022114670720/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767273.482969:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000800:00000100:0.0:1775767273.482974:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775767273.482975:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775767273.482978:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767273.482983:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4650
00000100:00080000:3.0:1775767273.482994:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767273.482998:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767273.483003:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775767273.483015:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744a80 x1862022114670848/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775767273.483025:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767273/real 1775767273]  req@ffff9f1485745500 x1862022114670336/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767298 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767273.483026:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767273.483035:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775767273.483039:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767273.483050:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4650
00000100:00100000:0.0:1775767273.483052:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745500 x1862022114670336/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767298 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767273.483057:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767273.483066:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767273.483067:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00080000:0.0:1775767273.483074:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767273.483077:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114670336:192.168.202.110@tcp:250:kworker.0
00000008:00100000:3.0:1775767273.483077:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767273.483087:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114670464:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767273.483121:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747480 x1862022114670976/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1775767273.483135:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775767273.483136:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767273.483141:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775767273.483143:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775767273.483147:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767273/real 1775767273]  req@ffff9f1485744e00 x1862022114670464/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767298 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767273.483151:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4650
00000100:00080000:3.0:1775767273.483159:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1775767273.483165:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744e00 x1862022114670464/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767298 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767273.483165:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767273.483173:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00080000:0.0:1775767273.483176:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767273.483180:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767273.483183:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114670464:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767273.483187:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745880 x1862022114671104/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767273.483194:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767273.483197:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775767273.483199:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767273.483202:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4650
00000100:00080000:3.0:1775767273.483205:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767273.483208:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775767273.483211:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114670592:192.168.202.110@tcp:38:kworker.0
00000008:00100000:3.0:1775767273.483211:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775767273.483222:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746300 x1862022114671232/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1775767273.483232:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767273.483242:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767273/real 1775767273]  req@ffff9f1485744380 x1862022114670592/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767298 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767273.483252:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744380 x1862022114670592/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767298 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767273.483265:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767273.483268:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767273.483270:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114670592:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767273.483275:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114670720:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767273.483293:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767273.483305:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767273/real 1775767273]  req@ffff9f1485746a00 x1862022114670720/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767298 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767273.483318:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746a00 x1862022114670720/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767298 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767273.483326:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767273.483330:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767273.483333:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114670720:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775767273.483339:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114670848:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767273.483360:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767273.483371:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767273/real 1775767273]  req@ffff9f1485744a80 x1862022114670848/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767298 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767273.483383:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744a80 x1862022114670848/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767298 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767273.483392:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767273.483395:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767273.483399:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114670848:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775767273.483405:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114670976:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767273.483426:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767273.483437:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767273/real 1775767273]  req@ffff9f1485747480 x1862022114670976/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767298 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767273.483450:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747480 x1862022114670976/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767298 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767273.483459:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767273.483462:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767273.483466:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114670976:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767273.483472:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114671104:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767273.483492:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767273.483503:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767273/real 1775767273]  req@ffff9f1485745880 x1862022114671104/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767298 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767273.483516:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745880 x1862022114671104/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767298 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767273.483524:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767273.483527:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767273.483531:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114671104:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767273.483549:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114671232:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767273.483603:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767273.483615:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767273/real 1775767273]  req@ffff9f1485746300 x1862022114671232/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767298 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767273.483628:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746300 x1862022114671232/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767298 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767273.483643:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767273.483647:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767273.483651:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114671232:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775767273.738730:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767274.506649:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767274.762777:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767275.530701:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767275.786749:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767276.554647:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767276.810788:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767277.578654:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767277.834756:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767278.602649:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:1.0:1775767278.602682:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767278.602689:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1775767278.602693:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767278.602701:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4656
00000100:00080000:1.0:1775767278.602712:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767278.602719:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775767278.602758:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff100 x1862022114671360/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767278.602785:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767278.602789:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775767278.602791:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767278.602795:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4656
00000100:00080000:1.0:1775767278.602799:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767278.602804:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767278.602815:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767278.602829:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fe300 x1862022114671488/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775767278.602836:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114671360:192.168.202.110@tcp:250:kworker.0
00000100:00080000:1.0:1775767278.602847:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767278.602851:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775767278.602853:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767278.602858:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4656
00000100:00080000:1.0:1775767278.602862:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767278.602866:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767278.602872:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767278.602883:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fc380 x1862022114671616/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767278.602895:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000800:00000100:0.0:1775767278.602900:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:1.0:1775767278.602902:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775767278.602905:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767278.602910:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4656
00000100:00080000:1.0:1775767278.602915:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767278.602920:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767278.602926:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775767278.602938:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fd880 x1862022114671744/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775767278.602945:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767278/real 1775767278]  req@ffff9f14ab4ff100 x1862022114671360/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767303 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767278.602953:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767278.602963:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1775767278.602966:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767278.602972:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4656
00000100:00100000:0.0:1775767278.602976:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff100 x1862022114671360/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767303 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767278.602978:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767278.602986:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767278.602995:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00080000:0.0:1775767278.602996:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767278.603004:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767278.603009:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114671360:192.168.202.110@tcp:250:kworker.0
00000100:00100000:1.0:1775767278.603014:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdc00 x1862022114671872/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767278.603024:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767278.603028:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775767278.603030:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775767278.603035:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114671488:192.168.202.110@tcp:38:kworker.0
00000100:00080000:1.0:1775767278.603037:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4656
00000100:00080000:1.0:1775767278.603045:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767278.603077:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000800:00000100:0.0:1775767278.603080:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000008:00100000:1.0:1775767278.603085:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767278.603096:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767278/real 1775767278]  req@ffff9f14ab4fe300 x1862022114671488/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767303 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767278.603102:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdf80 x1862022114672000/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775767278.603118:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fe300 x1862022114671488/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767303 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767278.603124:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767278.603132:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775767278.603134:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767278.603139:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4656
00000100:00080000:1.0:1775767278.603143:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767278.603150:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767278.603150:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767278.603156:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00080000:0.0:1775767278.603157:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767278.603164:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114671488:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775767278.603174:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fce00 x1862022114672128/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775767278.603175:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114671616:192.168.202.110@tcp:38:kworker.0
00000100:00080000:1.0:1775767278.603192:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767278.603196:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775767278.603198:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000800:00000100:0.0:1775767278.603203:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:1.0:1775767278.603207:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4656
00000100:00080000:1.0:1775767278.603213:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1775767278.603217:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767278/real 1775767278]  req@ffff9f14ab4fc380 x1862022114671616/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767303 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767278.603219:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767278.603228:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775767278.603239:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fc380 x1862022114671616/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767303 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767278.603245:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fea00 x1862022114672256/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767278.603263:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767278.603268:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767278.603271:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114671616:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767278.603280:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114671744:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767278.603299:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767278.603308:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767278/real 1775767278]  req@ffff9f14ab4fd880 x1862022114671744/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767303 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767278.603320:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fd880 x1862022114671744/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767303 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767278.603328:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767278.603331:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767278.603335:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114671744:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775767278.603340:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114671872:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767278.603353:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767278.603360:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767278/real 1775767278]  req@ffff9f14ab4fdc00 x1862022114671872/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767303 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767278.603372:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdc00 x1862022114671872/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767303 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767278.603380:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767278.603383:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767278.603386:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114671872:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775767278.603401:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114672000:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767278.603418:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767278.603426:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767278/real 1775767278]  req@ffff9f14ab4fdf80 x1862022114672000/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767303 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767278.603438:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdf80 x1862022114672000/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767303 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767278.603446:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767278.603449:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767278.603453:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114672000:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767278.603458:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114672128:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767278.603471:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767278.603479:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767278/real 1775767278]  req@ffff9f14ab4fce00 x1862022114672128/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767303 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767278.603489:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fce00 x1862022114672128/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767303 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767278.603503:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767278.603506:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767278.603509:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114672128:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767278.603514:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114672256:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767278.603529:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767278.603538:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767278/real 1775767278]  req@ffff9f14ab4fea00 x1862022114672256/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767303 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767278.603548:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fea00 x1862022114672256/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767303 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767278.603581:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767278.603584:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767278.603587:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114672256:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775767278.858691:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767279.626721:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767279.882744:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767280.650664:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767280.906747:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767281.674724:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767281.930805:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767282.698610:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767282.954764:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767283.722626:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:1.0:1775767283.722627:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767283.722634:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1775767283.722637:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767283.722653:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4661
00000100:00080000:1.0:1775767283.722663:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767283.722669:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775767283.722699:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fca80 x1862022114672384/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767283.722720:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767283.722723:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775767283.722724:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767283.722727:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4661
00000100:00080000:1.0:1775767283.722729:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767283.722732:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767283.722740:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767283.722749:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ffb80 x1862022114672512/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767283.722756:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767283.722759:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775767283.722760:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767283.722763:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4661
00000100:00080000:1.0:1775767283.722765:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767283.722767:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767283.722771:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767283.722776:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114672384:192.168.202.110@tcp:250:kworker.0
00000100:00100000:1.0:1775767283.722781:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fed80 x1862022114672640/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767283.722792:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767283.722794:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775767283.722795:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767283.722797:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4661
00000100:00080000:1.0:1775767283.722800:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767283.722802:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767283.722805:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775767283.722813:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff800 x1862022114672768/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767283.722820:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767283.722822:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1775767283.722823:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767283.722826:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4661
00000100:00080000:1.0:1775767283.722835:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767283.722837:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000800:00000100:0.0:1775767283.722842:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000008:00100000:1.0:1775767283.722843:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775767283.722859:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fea00 x1862022114672896/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767283.722870:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767283.722874:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00100000:0.0:1775767283.722877:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767283/real 1775767283]  req@ffff9f14ab4fca80 x1862022114672384/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767308 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767283.722877:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767283.722888:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4661
00000100:00080000:1.0:1775767283.722895:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1775767283.722899:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fca80 x1862022114672384/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767308 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767283.722902:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767283.722913:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00080000:0.0:1775767283.722915:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767283.722921:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767283.722924:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114672384:192.168.202.110@tcp:250:kworker.0
00000100:00100000:1.0:1775767283.722929:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fce00 x1862022114673024/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775767283.722933:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ffb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114672512:192.168.202.110@tcp:38:kworker.0
00000100:00080000:1.0:1775767283.722946:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767283.722949:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775767283.722951:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767283.722954:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4661
00000100:00080000:1.0:1775767283.722958:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000800:00000100:0.0:1775767283.722963:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:1.0:1775767283.722964:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767283.722970:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767283.722978:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdf80 x1862022114673152/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775767283.722984:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767283/real 1775767283]  req@ffff9f14ab4ffb80 x1862022114672512/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767308 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767283.722987:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767283.722992:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775767283.722994:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767283.723019:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4661
00000100:00080000:1.0:1775767283.723024:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767283.723028:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775767283.723032:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ffb80 x1862022114672512/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767308 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:1.0:1775767283.723032:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775767283.723044:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdc00 x1862022114673280/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767283.723046:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767283.723055:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767283.723060:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ffb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114672512:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767283.723081:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114672640:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767283.723098:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767283.723106:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767283/real 1775767283]  req@ffff9f14ab4fed80 x1862022114672640/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767308 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767283.723114:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fed80 x1862022114672640/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767308 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767283.723119:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767283.723121:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767283.723123:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114672640:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767283.723127:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114672768:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767283.723137:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767283.723142:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767283/real 1775767283]  req@ffff9f14ab4ff800 x1862022114672768/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767308 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767283.723150:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff800 x1862022114672768/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767308 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767283.723155:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767283.723157:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767283.723159:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114672768:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775767283.723163:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114672896:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767283.723172:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767283.723190:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767283/real 1775767283]  req@ffff9f14ab4fea00 x1862022114672896/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767308 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767283.723198:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fea00 x1862022114672896/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767308 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767283.723203:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767283.723205:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767283.723207:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114672896:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775767283.723210:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114673024:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767283.723221:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767283.723230:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767283/real 1775767283]  req@ffff9f14ab4fce00 x1862022114673024/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767308 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767283.723241:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fce00 x1862022114673024/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767308 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767283.723256:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767283.723260:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767283.723263:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114673024:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767283.723270:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114673152:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767283.723290:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767283.723301:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767283/real 1775767283]  req@ffff9f14ab4fdf80 x1862022114673152/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767308 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767283.723309:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdf80 x1862022114673152/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767308 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767283.723314:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767283.723316:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767283.723318:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114673152:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767283.723322:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114673280:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767283.723335:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767283.723341:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767283/real 1775767283]  req@ffff9f14ab4fdc00 x1862022114673280/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767308 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767283.723348:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdc00 x1862022114673280/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767308 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767283.723353:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767283.723355:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767283.723357:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114673280:192.168.202.110@tcp:8:kworker.0
00010000:00010000:3.0:1775767283.978741:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767284.746642:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767285.002712:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767285.770650:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767286.026736:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767286.794634:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767287.050717:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767287.818665:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767288.074743:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767288.842644:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:1.0:1775767288.842647:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767288.842654:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1775767288.842657:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767288.842666:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4666
00000100:00080000:1.0:1775767288.842675:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767288.842682:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775767288.842715:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fd880 x1862022114673408/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767288.842738:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767288.842741:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775767288.842742:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767288.842745:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4666
00000100:00080000:1.0:1775767288.842747:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767288.842750:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767288.842758:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767288.842767:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fc380 x1862022114673536/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767288.842775:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767288.842778:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775767288.842779:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767288.842784:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4666
00000100:00100000:0.0:1775767288.842785:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114673408:192.168.202.110@tcp:250:kworker.0
00000100:00080000:1.0:1775767288.842786:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767288.842793:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767288.842797:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767288.842804:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fe300 x1862022114673664/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767288.842811:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767288.842814:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775767288.842815:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767288.842818:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4666
00000100:00080000:1.0:1775767288.842833:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767288.842837:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767288.842842:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:0.0:1775767288.842845:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767288.842860:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff100 x1862022114673792/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767288.842878:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767288.842882:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1775767288.842884:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775767288.842889:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767288/real 1775767288]  req@ffff9f14ab4fd880 x1862022114673408/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767313 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767288.842890:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4666
00000100:00080000:1.0:1775767288.842896:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767288.842899:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767288.842903:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775767288.842913:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdc00 x1862022114673920/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775767288.842926:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fd880 x1862022114673408/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767313 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767288.842932:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767288.842937:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767288.842937:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775767288.842943:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767288.842946:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767288.842962:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114673408:192.168.202.110@tcp:250:kworker.0
00000100:00080000:1.0:1775767288.842965:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4666
00000100:00100000:0.0:1775767288.842972:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114673536:192.168.202.110@tcp:38:kworker.0
00000100:00080000:1.0:1775767288.842972:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767288.842980:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767288.842987:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767288.843001:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdf80 x1862022114674048/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1775767288.843009:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:1.0:1775767288.843013:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767288.843016:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775767288.843018:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767288.843021:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4666
00000100:00080000:1.0:1775767288.843026:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1775767288.843028:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767288/real 1775767288]  req@ffff9f14ab4fc380 x1862022114673536/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767313 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767288.843031:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775767288.843045:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fc380 x1862022114673536/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767313 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:1.0:1775767288.843050:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00080000:0.0:1775767288.843052:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767288.843057:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767288.843060:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114673536:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775767288.843068:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fce00 x1862022114674176/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775767288.843081:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114673664:192.168.202.110@tcp:38:kworker.0
00000100:00080000:1.0:1775767288.843081:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767288.843091:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775767288.843093:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767288.843099:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4666
00000800:00000100:0.0:1775767288.843105:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:1.0:1775767288.843105:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767288.843114:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775767288.843118:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767288/real 1775767288]  req@ffff9f14ab4fe300 x1862022114673664/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767313 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:1.0:1775767288.843120:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775767288.843133:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fe300 x1862022114673664/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767313 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767288.843140:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00100000:1.0:1775767288.843140:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fea00 x1862022114674304/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767288.843146:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767288.843152:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114673664:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767288.843162:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114673792:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767288.843179:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767288.843191:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767288/real 1775767288]  req@ffff9f14ab4ff100 x1862022114673792/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767313 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767288.843204:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff100 x1862022114673792/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767313 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767288.843212:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767288.843215:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767288.843219:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114673792:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775767288.843225:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114673920:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767288.843241:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767288.843248:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767288/real 1775767288]  req@ffff9f14ab4fdc00 x1862022114673920/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767313 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767288.843255:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdc00 x1862022114673920/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767313 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767288.843261:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767288.843263:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767288.843265:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114673920:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775767288.843269:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114674048:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767288.843278:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767288.843283:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767288/real 1775767288]  req@ffff9f14ab4fdf80 x1862022114674048/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767313 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767288.843290:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdf80 x1862022114674048/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767313 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767288.843295:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767288.843297:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767288.843299:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114674048:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767288.843309:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114674176:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767288.843319:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767288.843325:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767288/real 1775767288]  req@ffff9f14ab4fce00 x1862022114674176/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767313 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767288.843332:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fce00 x1862022114674176/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767313 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767288.843336:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767288.843338:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767288.843340:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114674176:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767288.843344:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114674304:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767288.843353:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767288.843358:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767288/real 1775767288]  req@ffff9f14ab4fea00 x1862022114674304/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767313 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767288.843365:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fea00 x1862022114674304/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767313 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767288.843370:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767288.843371:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767288.843373:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114674304:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775767289.098731:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767289.866659:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767290.122696:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767290.890638:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767291.146710:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767291.914660:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767292.170729:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767292.938621:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767293.194694:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767293.962651:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:0.0:1775767293.962668:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767293.962673:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1775767293.962677:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767293.962685:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4671
00000100:00080000:0.0:1775767293.962694:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767293.962699:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775767293.962731:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d1180 x1862022114674432/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767293.962755:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767293.962757:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775767293.962759:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767293.962761:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4671
00000100:00080000:0.0:1775767293.962764:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767293.962767:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767293.962775:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767293.962784:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3480 x1862022114674560/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:1.0:1775767293.962788:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114674432:192.168.202.110@tcp:250:kworker.0
00000100:00080000:0.0:1775767293.962801:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767293.962811:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:0.0:1775767293.962813:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767293.962816:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4671
00000100:00080000:0.0:1775767293.962818:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767293.962821:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767293.962825:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767293.962832:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0380 x1862022114674688/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767293.962839:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767293.962841:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775767293.962842:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767293.962844:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4671
00000100:00080000:0.0:1775767293.962846:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767293.962849:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767293.962853:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:1.0:1775767293.962854:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767293.962865:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d1f80 x1862022114674816/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767293.962872:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767293.962875:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1775767293.962876:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767293.962878:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4671
00000100:00080000:0.0:1775767293.962881:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767293.962883:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767293.962886:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775767293.962899:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3b80 x1862022114674944/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:1.0:1775767293.962902:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767293/real 1775767293]  req@ffff9f14a04d1180 x1862022114674432/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767318 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767293.962926:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767293.962934:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775767293.962936:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767293.962940:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4671
00000100:00080000:0.0:1775767293.962944:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767293.962947:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767293.962951:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767293.962953:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1180 x1862022114674432/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767318 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767293.962963:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0e00 x1862022114675072/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767293.962963:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767293.962970:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767293.962977:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114674432:192.168.202.110@tcp:250:kworker.0
00000100:00080000:0.0:1775767293.962979:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767293.962986:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:0.0:1775767293.962987:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767293.962991:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4671
00000100:00080000:0.0:1775767293.962993:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767293.962995:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767293.962999:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767293.963003:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114674560:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767293.963008:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0a80 x1862022114675200/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767293.963016:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767293.963019:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775767293.963020:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767293.963022:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4671
00000100:00080000:0.0:1775767293.963025:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000800:00000100:1.0:1775767293.963026:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:0.0:1775767293.963030:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767293.963034:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775767293.963041:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767293/real 1775767293]  req@ffff9f14a04d3480 x1862022114674560/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767318 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767293.963044:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d2a00 x1862022114675328/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:1.0:1775767293.963064:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3480 x1862022114674560/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767318 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767293.963071:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767293.963074:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767293.963078:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114674560:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775767293.963083:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114674688:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775767293.963094:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767293.963100:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767293/real 1775767293]  req@ffff9f14a04d0380 x1862022114674688/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767318 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767293.963107:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0380 x1862022114674688/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767318 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767293.963112:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767293.963114:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767293.963116:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114674688:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775767293.963119:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114674816:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775767293.963129:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767293.963135:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767293/real 1775767293]  req@ffff9f14a04d1f80 x1862022114674816/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767318 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767293.963142:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1f80 x1862022114674816/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767318 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767293.963147:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767293.963149:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767293.963150:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114674816:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775767293.963154:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114674944:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775767293.963162:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767293.963168:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767293/real 1775767293]  req@ffff9f14a04d3b80 x1862022114674944/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767318 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767293.963174:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3b80 x1862022114674944/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767318 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767293.963180:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767293.963181:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767293.963183:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114674944:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775767293.963186:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114675072:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775767293.963195:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767293.963199:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767293/real 1775767293]  req@ffff9f14a04d0e00 x1862022114675072/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767318 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767293.963206:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0e00 x1862022114675072/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767318 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767293.963215:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767293.963217:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767293.963219:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114675072:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775767293.963231:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114675200:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775767293.963240:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767293.963246:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767293/real 1775767293]  req@ffff9f14a04d0a80 x1862022114675200/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767318 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767293.963252:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0a80 x1862022114675200/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767318 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767293.963257:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767293.963259:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767293.963261:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114675200:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775767293.963264:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114675328:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775767293.963273:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767293.963278:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767293/real 1775767293]  req@ffff9f14a04d2a00 x1862022114675328/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767318 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767293.963285:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d2a00 x1862022114675328/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767318 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767293.963289:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767293.963291:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767293.963293:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114675328:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775767294.218700:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767294.986626:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767295.242692:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767296.010624:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767296.266776:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767297.034772:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767297.290714:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767298.058688:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767298.314798:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1775767299.082683:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767299.082692:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1775767299.082697:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767299.082705:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4676
00000100:00080000:0.0:1775767299.082716:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767299.082723:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775767299.082738:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:0.0:1775767299.082760:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3100 x1862022114675456/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767299.082783:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767299.082788:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775767299.082791:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767299.082796:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4676
00000100:00080000:0.0:1775767299.082800:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767299.082803:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767299.082813:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767299.082835:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d2300 x1862022114675584/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:1.0:1775767299.082836:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114675456:192.168.202.110@tcp:250:kworker.0
00000100:00080000:0.0:1775767299.082846:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767299.082849:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:0.0:1775767299.082851:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767299.082854:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4676
00000100:00080000:0.0:1775767299.082873:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767299.082876:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767299.082881:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767299.082891:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0000 x1862022114675712/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767299.082899:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767299.082901:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775767299.082903:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767299.082906:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4676
00000100:00080000:0.0:1775767299.082908:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767299.082912:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767299.082916:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:1.0:1775767299.082918:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767299.082924:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d1500 x1862022114675840/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767299.082932:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767299.082934:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1775767299.082936:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767299.082939:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4676
00000100:00080000:0.0:1775767299.082941:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767299.082944:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767299.082948:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775767299.082956:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3800 x1862022114675968/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:1.0:1775767299.082957:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767299/real 1775767299]  req@ffff9f14a04d3100 x1862022114675456/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767324 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767299.082966:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767299.082970:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775767299.082971:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767299.082974:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4676
00000100:00100000:1.0:1775767299.082974:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3100 x1862022114675456/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767324 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767299.082978:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767299.082982:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00080000:1.0:1775767299.082984:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000008:00100000:0.0:1775767299.082986:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00080000:1.0:1775767299.082988:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767299.082992:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114675456:192.168.202.110@tcp:250:kworker.0
00000100:00100000:0.0:1775767299.082995:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d2a00 x1862022114676096/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767299.083003:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767299.083006:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:0.0:1775767299.083007:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767299.083011:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4676
00000100:00100000:1.0:1775767299.083012:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114675584:192.168.202.110@tcp:38:kworker.0
00000100:00080000:0.0:1775767299.083014:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767299.083017:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767299.083022:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767299.083030:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0a80 x1862022114676224/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767299.083040:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000800:00000100:1.0:1775767299.083043:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:0.0:1775767299.083044:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775767299.083046:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767299.083050:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4676
00000100:00080000:0.0:1775767299.083053:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:1.0:1775767299.083055:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767299/real 1775767299]  req@ffff9f14a04d2300 x1862022114675584/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767324 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767299.083057:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767299.083061:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775767299.083068:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d2300 x1862022114675584/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767324 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767299.083071:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0e00 x1862022114676352/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767299.083078:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767299.083082:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767299.083085:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114675584:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775767299.083091:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114675712:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775767299.083109:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767299.083120:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767299/real 1775767299]  req@ffff9f14a04d0000 x1862022114675712/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767324 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767299.083132:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0000 x1862022114675712/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767324 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767299.083137:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767299.083139:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767299.083142:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114675712:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775767299.083146:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114675840:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775767299.083160:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767299.083167:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767299/real 1775767299]  req@ffff9f14a04d1500 x1862022114675840/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767324 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767299.083174:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1500 x1862022114675840/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767324 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767299.083179:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767299.083180:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767299.083182:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114675840:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775767299.083185:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114675968:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775767299.083195:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767299.083200:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767299/real 1775767299]  req@ffff9f14a04d3800 x1862022114675968/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767324 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767299.083207:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3800 x1862022114675968/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767324 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767299.083212:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767299.083214:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767299.083216:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114675968:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775767299.083219:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114676096:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775767299.083228:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767299.083234:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767299/real 1775767299]  req@ffff9f14a04d2a00 x1862022114676096/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767324 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767299.083240:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d2a00 x1862022114676096/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767324 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767299.083245:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767299.083247:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767299.083249:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114676096:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775767299.083262:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114676224:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775767299.083272:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767299.083277:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767299/real 1775767299]  req@ffff9f14a04d0a80 x1862022114676224/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767324 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767299.083284:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0a80 x1862022114676224/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767324 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767299.083288:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767299.083290:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767299.083293:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114676224:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775767299.083296:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114676352:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775767299.083306:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767299.083310:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767299/real 1775767299]  req@ffff9f14a04d0e00 x1862022114676352/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767324 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767299.083317:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0e00 x1862022114676352/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767324 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767299.083322:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767299.083324:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767299.083325:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114676352:192.168.202.110@tcp:8:kworker.0
00010000:00010000:3.0:1775767299.338691:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767300.106679:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767300.362754:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767301.130709:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767301.386754:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767302.154683:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767302.410707:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767303.178713:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767303.434794:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1775767304.202688:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:2.0:1775767304.202699:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:0.0:1775767304.202700:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1775767304.202706:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767304.202718:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4681
00000100:00080000:0.0:1775767304.202732:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767304.202739:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775767304.202784:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3b80 x1862022114676480/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767304.202806:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767304.202828:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775767304.202831:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767304.202837:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4681
00000100:00080000:0.0:1775767304.202842:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767304.202848:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767304.202862:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767304.202873:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114676480:192.168.202.110@tcp:250:kworker.0
00000100:00100000:0.0:1775767304.202886:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d1f80 x1862022114676608/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767304.202897:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767304.202901:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:0.0:1775767304.202904:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767304.202909:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4681
00000100:00080000:0.0:1775767304.202916:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767304.202920:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767304.202926:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767304.202942:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0380 x1862022114676736/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767304.202951:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767304.202955:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775767304.202958:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000800:00000100:1.0:1775767304.202967:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:0.0:1775767304.202972:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4681
00000100:00080000:0.0:1775767304.202976:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767304.202980:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767304.202986:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775767304.202998:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3480 x1862022114676864/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767304.203007:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767304.203011:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1775767304.203013:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767304.203018:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4681
00000100:00100000:1.0:1775767304.203023:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767304/real 1775767304]  req@ffff9f14a04d3b80 x1862022114676480/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767329 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767304.203025:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767304.203033:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767304.203048:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775767304.203058:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3b80 x1862022114676480/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767329 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767304.203068:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d1180 x1862022114676992/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767304.203073:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767304.203090:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767304.203103:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114676480:192.168.202.110@tcp:250:kworker.0
00000100:00080000:0.0:1775767304.203106:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767304.203110:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775767304.203112:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767304.203120:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4681
00000100:00080000:0.0:1775767304.203124:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767304.203128:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767304.203133:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767304.203135:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114676608:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767304.203148:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3b80 x1862022114677120/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767304.203158:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767304.203161:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:0.0:1775767304.203162:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767304.203167:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4681
00000100:00080000:0.0:1775767304.203171:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767304.203182:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000800:00000100:1.0:1775767304.203183:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000008:00100000:0.0:1775767304.203195:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767304.203209:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0e00 x1862022114677248/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767304.203216:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:1.0:1775767304.203220:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767304/real 1775767304]  req@ffff9f14a04d1f80 x1862022114676608/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767329 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767304.203221:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775767304.203225:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767304.203241:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4681
00000100:00080000:0.0:1775767304.203248:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767304.203251:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775767304.203257:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1f80 x1862022114676608/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767329 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:0.0:1775767304.203263:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00080000:1.0:1775767304.203280:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00100000:0.0:1775767304.203290:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0a80 x1862022114677376/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767304.203291:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767304.203305:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114676608:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775767304.203322:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114676736:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775767304.203351:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767304.203375:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767304/real 1775767304]  req@ffff9f14a04d0380 x1862022114676736/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767329 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767304.203401:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0380 x1862022114676736/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767329 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767304.203407:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767304.203414:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767304.203418:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114676736:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775767304.203423:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114676864:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775767304.203437:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767304.203444:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767304/real 1775767304]  req@ffff9f14a04d3480 x1862022114676864/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767329 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767304.203451:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3480 x1862022114676864/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767329 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767304.203456:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767304.203459:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767304.203461:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114676864:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775767304.203464:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114676992:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775767304.203473:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767304.203488:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767304/real 1775767304]  req@ffff9f14a04d1180 x1862022114676992/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767329 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767304.203495:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1180 x1862022114676992/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767329 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767304.203500:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767304.203502:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767304.203503:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114676992:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775767304.203518:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114677120:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775767304.203530:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767304.203536:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767304/real 1775767304]  req@ffff9f14a04d3b80 x1862022114677120/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767329 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767304.203543:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3b80 x1862022114677120/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767329 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767304.203548:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767304.203550:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767304.203578:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114677120:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775767304.203584:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114677248:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775767304.203599:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767304.203606:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767304/real 1775767304]  req@ffff9f14a04d0e00 x1862022114677248/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767329 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767304.203615:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0e00 x1862022114677248/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767329 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767304.203620:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767304.203622:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767304.203624:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114677248:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775767304.203629:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114677376:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775767304.203648:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767304.203664:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767304/real 1775767304]  req@ffff9f14a04d0a80 x1862022114677376/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767329 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767304.203677:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0a80 x1862022114677376/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767329 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767304.203685:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767304.203689:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767304.203692:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114677376:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775767304.458765:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767305.226711:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767305.482794:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767306.250688:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:0.0:1775767306.506824:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767307.274747:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767307.530746:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767308.298686:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767308.554737:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767309.322669:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:1.0:1775767309.322714:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767309.322725:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1775767309.322731:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767309.322743:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4686
00000100:00080000:1.0:1775767309.322757:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767309.322790:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775767309.322838:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff800 x1862022114677504/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767309.322872:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767309.322876:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775767309.322878:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767309.322884:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4686
00000100:00080000:1.0:1775767309.322888:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767309.322893:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767309.322906:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767309.322911:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114677504:192.168.202.110@tcp:250:kworker.0
00000100:00100000:1.0:1775767309.322922:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fed80 x1862022114677632/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767309.322935:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767309.322939:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775767309.322941:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767309.322946:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4686
00000100:00080000:1.0:1775767309.322951:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767309.322954:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767309.322960:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767309.322970:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ffb80 x1862022114677760/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1775767309.322986:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:1.0:1775767309.322993:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767309.322997:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775767309.322999:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767309.323004:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4686
00000100:00080000:1.0:1775767309.323010:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767309.323013:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767309.323019:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775767309.323033:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fca80 x1862022114677888/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775767309.323036:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767309/real 1775767309]  req@ffff9f14ab4ff800 x1862022114677504/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767334 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767309.323062:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767309.323066:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00100000:0.0:1775767309.323069:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff800 x1862022114677504/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767334 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767309.323069:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767309.323084:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4686
00000100:00080000:1.0:1775767309.323090:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767309.323094:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767309.323095:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767309.323103:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00080000:0.0:1775767309.323105:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767309.323112:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114677504:192.168.202.110@tcp:250:kworker.0
00000100:00100000:1.0:1775767309.323119:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fea00 x1862022114678016/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767309.323130:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767309.323134:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775767309.323136:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767309.323141:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4686
00000100:00100000:0.0:1775767309.323144:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114677632:192.168.202.110@tcp:38:kworker.0
00000100:00080000:1.0:1775767309.323148:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767309.323155:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767309.323162:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000800:00000100:0.0:1775767309.323171:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767309.323176:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fce00 x1862022114678144/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775767309.323185:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767309/real 1775767309]  req@ffff9f14ab4fed80 x1862022114677632/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767334 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767309.323195:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767309.323199:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00100000:0.0:1775767309.323202:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fed80 x1862022114677632/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767334 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767309.323203:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767309.323212:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4686
00000100:00080000:0.0:1775767309.323219:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767309.323220:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767309.323227:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00080000:1.0:1775767309.323228:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767309.323235:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767309.323241:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114677632:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775767309.323247:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdf80 x1862022114678272/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775767309.323248:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ffb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114677760:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767309.323269:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:1.0:1775767309.323274:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1775767309.323278:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767309/real 1775767309]  req@ffff9f14ab4ffb80 x1862022114677760/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767334 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767309.323288:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775767309.323292:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775767309.323295:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ffb80 x1862022114677760/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767334 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767309.323301:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4686
00000100:00080000:0.0:1775767309.323306:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767309.323309:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767309.323313:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767309.323318:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ffb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114677760:192.168.202.110@tcp:38:kworker.0
00000100:00080000:1.0:1775767309.323318:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767309.323327:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775767309.323331:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114677888:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775767309.323343:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdc00 x1862022114678400/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1775767309.323359:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767309.323377:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767309/real 1775767309]  req@ffff9f14ab4fca80 x1862022114677888/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767334 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767309.323391:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fca80 x1862022114677888/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767334 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767309.323397:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767309.323400:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767309.323403:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114677888:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775767309.323409:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114678016:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767309.323430:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767309.323441:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767309/real 1775767309]  req@ffff9f14ab4fea00 x1862022114678016/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767334 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767309.323455:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fea00 x1862022114678016/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767334 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767309.323463:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767309.323466:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767309.323469:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114678016:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775767309.323483:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114678144:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767309.323498:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767309.323505:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767309/real 1775767309]  req@ffff9f14ab4fce00 x1862022114678144/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767334 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767309.323514:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fce00 x1862022114678144/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767334 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767309.323519:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767309.323521:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767309.323523:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114678144:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767309.323526:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114678272:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767309.323535:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767309.323543:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767309/real 1775767309]  req@ffff9f14ab4fdf80 x1862022114678272/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767334 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767309.323581:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdf80 x1862022114678272/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767334 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767309.323591:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767309.323595:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767309.323598:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114678272:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767309.323603:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114678400:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767309.323620:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767309.323627:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767309/real 1775767309]  req@ffff9f14ab4fdc00 x1862022114678400/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767334 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767309.323635:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdc00 x1862022114678400/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767334 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767309.323645:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767309.323648:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767309.323651:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114678400:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775767309.578748:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767310.346614:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767310.602706:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767311.370667:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767311.626726:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767312.394598:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767312.650721:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767313.418625:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:0.0:1775767313.674700:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1775767314.442645:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:2.0:1775767314.442650:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:1.0:1775767314.442652:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1775767314.442658:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767314.442665:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4691
00000100:00080000:1.0:1775767314.442674:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767314.442680:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775767314.442731:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff100 x1862022114678528/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767314.442754:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767314.442757:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775767314.442758:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767314.442761:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4691
00000100:00080000:1.0:1775767314.442764:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767314.442767:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767314.442775:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767314.442786:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fe300 x1862022114678656/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767314.442795:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767314.442797:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775767314.442800:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775767314.442802:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114678528:192.168.202.110@tcp:250:kworker.0
00000100:00080000:1.0:1775767314.442814:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4691
00000100:00080000:1.0:1775767314.442817:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767314.442819:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767314.442824:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767314.442833:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fc380 x1862022114678784/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767314.442840:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767314.442842:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775767314.442844:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767314.442846:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4691
00000100:00080000:1.0:1775767314.442849:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767314.442851:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767314.442854:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:0.0:1775767314.442857:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767314.442865:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fd880 x1862022114678912/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767314.442871:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767314.442873:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1775767314.442874:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767314.442876:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4691
00000100:00080000:1.0:1775767314.442880:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767314.442882:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767314.442885:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775767314.442891:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdc00 x1862022114679040/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767314.442897:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767314.442899:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775767314.442902:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767314.442905:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4691
00000100:00080000:1.0:1775767314.442908:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1775767314.442911:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767314/real 1775767314]  req@ffff9f14ab4ff100 x1862022114678528/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767339 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767314.442922:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767314.442932:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767314.442939:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff100 x1862022114678528/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767339 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767314.442946:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdf80 x1862022114679168/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767314.442952:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767314.442960:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767314.442966:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114678528:192.168.202.110@tcp:250:kworker.0
00000100:00080000:1.0:1775767314.442967:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767314.442973:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775767314.442975:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767314.442981:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4691
00000100:00080000:1.0:1775767314.442984:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1775767314.442989:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114678656:192.168.202.110@tcp:38:kworker.0
00000100:00080000:1.0:1775767314.442990:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767314.442999:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767314.443012:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fce00 x1862022114679296/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1775767314.443021:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:1.0:1775767314.443022:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767314.443026:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775767314.443028:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767314.443032:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4691
00000100:00080000:1.0:1775767314.443035:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1775767314.443037:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767314/real 1775767314]  req@ffff9f14ab4fe300 x1862022114678656/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767339 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767314.443041:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767314.443047:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775767314.443053:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fe300 x1862022114678656/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767339 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767314.443058:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fea00 x1862022114679424/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767314.443071:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767314.443077:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767314.443116:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114678656:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767314.443121:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114678784:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767314.443146:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767314.443158:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767314/real 1775767314]  req@ffff9f14ab4fc380 x1862022114678784/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767339 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767314.443171:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fc380 x1862022114678784/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767339 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767314.443181:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767314.443184:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767314.443188:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114678784:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767314.443195:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114678912:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767314.443212:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767314.443220:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767314/real 1775767314]  req@ffff9f14ab4fd880 x1862022114678912/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767339 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767314.443228:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fd880 x1862022114678912/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767339 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767314.443234:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767314.443236:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767314.443238:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114678912:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775767314.443242:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114679040:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767314.443251:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767314.443257:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767314/real 1775767314]  req@ffff9f14ab4fdc00 x1862022114679040/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767339 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767314.443264:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdc00 x1862022114679040/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767339 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767314.443269:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767314.443280:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767314.443283:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114679040:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775767314.443289:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114679168:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767314.443308:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767314.443319:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767314/real 1775767314]  req@ffff9f14ab4fdf80 x1862022114679168/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767339 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767314.443332:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdf80 x1862022114679168/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767339 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767314.443340:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767314.443344:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767314.443346:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114679168:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767314.443360:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114679296:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767314.443375:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767314.443382:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767314/real 1775767314]  req@ffff9f14ab4fce00 x1862022114679296/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767339 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767314.443389:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fce00 x1862022114679296/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767339 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767314.443395:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767314.443397:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767314.443400:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114679296:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767314.443403:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114679424:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767314.443412:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767314.443418:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767314/real 1775767314]  req@ffff9f14ab4fea00 x1862022114679424/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767339 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767314.443425:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fea00 x1862022114679424/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767339 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767314.443433:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767314.443437:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767314.443441:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114679424:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775767314.698680:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767315.466647:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767315.722761:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767316.490672:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767316.746750:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767317.514630:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767317.770829:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767318.538735:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767318.794723:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767319.562645:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:1.0:1775767319.562648:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767319.562658:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1775767319.562662:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767319.562679:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4697
00000100:00080000:1.0:1775767319.562689:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767319.562695:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775767319.562728:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fca80 x1862022114679552/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767319.562747:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767319.562750:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775767319.562751:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767319.562754:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4697
00000100:00080000:1.0:1775767319.562756:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767319.562759:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767319.562767:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767319.562776:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ffb80 x1862022114679680/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775767319.562781:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114679552:192.168.202.110@tcp:250:kworker.0
00000100:00080000:1.0:1775767319.562789:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767319.562791:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775767319.562792:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767319.562795:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4697
00000100:00080000:1.0:1775767319.562797:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767319.562800:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767319.562804:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767319.562813:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fed80 x1862022114679808/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767319.562819:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767319.562822:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775767319.562823:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767319.562825:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4697
00000100:00080000:1.0:1775767319.562833:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767319.562835:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767319.562839:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:0.0:1775767319.562843:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767319.562850:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff800 x1862022114679936/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767319.562857:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767319.562859:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1775767319.562860:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767319.562862:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4697
00000100:00080000:1.0:1775767319.562865:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767319.562867:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767319.562870:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775767319.562879:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767319/real 1775767319]  req@ffff9f14ab4fca80 x1862022114679552/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767344 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767319.562880:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fea00 x1862022114680064/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767319.562912:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767319.562916:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775767319.562917:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767319.562922:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4697
00000100:00100000:0.0:1775767319.562923:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fca80 x1862022114679552/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767344 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767319.562924:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767319.562930:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767319.562935:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767319.562944:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fce00 x1862022114680192/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767319.562946:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767319.562952:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767319.562955:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:0.0:1775767319.562957:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:1.0:1775767319.562958:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767319.562964:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4697
00000100:00100000:0.0:1775767319.562968:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114679552:192.168.202.110@tcp:250:kworker.0
00000100:00080000:1.0:1775767319.562969:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767319.562976:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767319.562980:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767319.562989:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdf80 x1862022114680320/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775767319.562998:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ffb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114679680:192.168.202.110@tcp:38:kworker.0
00000100:00080000:1.0:1775767319.563001:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767319.563006:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775767319.563008:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767319.563013:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4697
00000100:00080000:1.0:1775767319.563019:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000800:00000100:0.0:1775767319.563023:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:1.0:1775767319.563028:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767319.563035:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775767319.563036:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767319/real 1775767319]  req@ffff9f14ab4ffb80 x1862022114679680/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767344 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767319.563049:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ffb80 x1862022114679680/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767344 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767319.563053:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdc00 x1862022114680448/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767319.563065:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767319.563071:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767319.563076:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ffb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114679680:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767319.563080:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114679808:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767319.563094:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767319.563100:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767319/real 1775767319]  req@ffff9f14ab4fed80 x1862022114679808/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767344 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767319.563108:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fed80 x1862022114679808/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767344 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767319.563112:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767319.563114:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767319.563116:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114679808:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767319.563120:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114679936:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767319.563130:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767319.563135:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767319/real 1775767319]  req@ffff9f14ab4ff800 x1862022114679936/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767344 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767319.563142:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff800 x1862022114679936/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767344 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767319.563147:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767319.563149:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767319.563151:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114679936:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775767319.563154:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114680064:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767319.563164:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767319.563169:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767319/real 1775767319]  req@ffff9f14ab4fea00 x1862022114680064/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767344 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767319.563175:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fea00 x1862022114680064/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767344 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767319.563180:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767319.563182:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767319.563183:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114680064:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775767319.563187:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114680192:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767319.563195:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767319.563201:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767319/real 1775767319]  req@ffff9f14ab4fce00 x1862022114680192/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767344 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767319.563213:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fce00 x1862022114680192/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767344 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767319.563217:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767319.563219:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767319.563221:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114680192:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767319.563232:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114680320:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767319.563241:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767319.563246:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767319/real 1775767319]  req@ffff9f14ab4fdf80 x1862022114680320/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767344 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767319.563254:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdf80 x1862022114680320/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767344 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767319.563258:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767319.563260:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767319.563262:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114680320:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767319.563265:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114680448:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767319.563274:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767319.563279:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767319/real 1775767319]  req@ffff9f14ab4fdc00 x1862022114680448/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767344 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767319.563286:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdc00 x1862022114680448/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767344 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767319.563295:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767319.563298:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767319.563299:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114680448:192.168.202.110@tcp:8:kworker.0
00010000:00010000:3.0:1775767319.818777:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767320.586732:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:0.0:1775767320.842721:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767321.610703:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767321.866731:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767322.634672:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767322.890777:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767323.658677:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767323.914727:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1775767324.682676:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1775767324.682679:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:1.0:1775767324.682686:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1775767324.682690:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767324.682698:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4702
00000100:00080000:1.0:1775767324.682707:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767324.682713:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775767324.682746:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fd880 x1862022114680576/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767324.682766:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767324.682769:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775767324.682770:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767324.682773:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4702
00000100:00080000:1.0:1775767324.682775:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767324.682778:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767324.682785:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767324.682796:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fc380 x1862022114680704/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775767324.682802:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114680576:192.168.202.110@tcp:250:kworker.0
00000100:00080000:1.0:1775767324.682806:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767324.682812:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775767324.682814:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767324.682818:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4702
00000100:00080000:1.0:1775767324.682823:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767324.682827:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767324.682832:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767324.682843:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fe300 x1862022114680832/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767324.682854:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767324.682858:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775767324.682860:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767324.682863:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4702
00000100:00080000:1.0:1775767324.682871:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767324.682876:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000800:00000100:0.0:1775767324.682881:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000008:00100000:1.0:1775767324.682886:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775767324.682901:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff100 x1862022114680960/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767324.682910:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767324.682914:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1775767324.682915:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767324.682920:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4702
00000100:00100000:0.0:1775767324.682924:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767324/real 1775767324]  req@ffff9f14ab4fd880 x1862022114680576/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767349 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767324.682926:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767324.682935:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767324.682943:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775767324.682947:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fd880 x1862022114680576/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767349 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767324.682957:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00100000:1.0:1775767324.682958:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdc00 x1862022114681088/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767324.682964:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767324.682972:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114680576:192.168.202.110@tcp:250:kworker.0
00000100:00080000:1.0:1775767324.682985:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767324.682988:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775767324.682990:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767324.682995:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4702
00000100:00100000:0.0:1775767324.682998:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114680704:192.168.202.110@tcp:38:kworker.0
00000100:00080000:1.0:1775767324.683000:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767324.683008:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767324.683013:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000800:00000100:0.0:1775767324.683020:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767324.683029:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdf80 x1862022114681216/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775767324.683032:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767324/real 1775767324]  req@ffff9f14ab4fc380 x1862022114680704/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767349 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767324.683053:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fc380 x1862022114680704/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767349 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767324.683061:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767324.683063:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767324.683067:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:0.0:1775767324.683071:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00080000:1.0:1775767324.683072:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775767324.683079:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114680704:192.168.202.110@tcp:38:kworker.0
00000100:00080000:1.0:1775767324.683080:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4702
00000100:00080000:1.0:1775767324.683089:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1775767324.683093:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114680832:192.168.202.110@tcp:38:kworker.0
00000100:00080000:1.0:1775767324.683096:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767324.683106:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767324.683121:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fce00 x1862022114681344/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1775767324.683128:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:1.0:1775767324.683139:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1775767324.683140:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767324/real 1775767324]  req@ffff9f14ab4fe300 x1862022114680832/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767349 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767324.683146:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775767324.683150:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767324.683158:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4702
00000100:00100000:0.0:1775767324.683162:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fe300 x1862022114680832/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767349 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767324.683164:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767324.683171:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00080000:0.0:1775767324.683174:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000008:00100000:1.0:1775767324.683177:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00080000:0.0:1775767324.683180:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767324.683185:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114680832:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767324.683191:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114680960:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775767324.683195:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fea00 x1862022114681472/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1775767324.683211:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767324.683218:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767324/real 1775767324]  req@ffff9f14ab4ff100 x1862022114680960/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767349 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767324.683227:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff100 x1862022114680960/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767349 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767324.683238:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767324.683240:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767324.683242:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114680960:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775767324.683246:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114681088:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767324.683255:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767324.683260:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767324/real 1775767324]  req@ffff9f14ab4fdc00 x1862022114681088/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767349 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767324.683267:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdc00 x1862022114681088/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767349 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767324.683273:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767324.683275:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767324.683276:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114681088:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775767324.683288:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114681216:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767324.683307:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767324.683314:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767324/real 1775767324]  req@ffff9f14ab4fdf80 x1862022114681216/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767349 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767324.683322:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdf80 x1862022114681216/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767349 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767324.683328:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767324.683330:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767324.683332:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114681216:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767324.683336:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114681344:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767324.683347:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767324.683352:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767324/real 1775767324]  req@ffff9f14ab4fce00 x1862022114681344/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767349 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767324.683359:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fce00 x1862022114681344/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767349 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767324.683364:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767324.683366:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767324.683367:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114681344:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767324.683371:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114681472:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767324.683380:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767324.683385:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767324/real 1775767324]  req@ffff9f14ab4fea00 x1862022114681472/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767349 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767324.683393:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fea00 x1862022114681472/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767349 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767324.683404:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767324.683407:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767324.683409:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114681472:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775767324.938776:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767325.706673:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767325.962719:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767326.730674:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767326.986752:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767327.754664:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:0.0:1775767328.010717:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767328.778673:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767329.034836:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767329.802643:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:2.0:1775767329.802681:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767329.802686:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1775767329.802690:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767329.802698:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4707
00000100:00080000:2.0:1775767329.802708:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767329.802713:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775767329.802744:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3800 x1862022114681600/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767329.802765:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767329.802768:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775767329.802769:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767329.802773:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4707
00000100:00080000:2.0:1775767329.802778:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767329.802782:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767329.802793:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767329.802795:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114681600:192.168.202.110@tcp:250:kworker.0
00000100:00100000:2.0:1775767329.802809:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0700 x1862022114681728/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767329.802823:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767329.802827:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775767329.802829:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767329.802834:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4707
00000100:00080000:2.0:1775767329.802838:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767329.802841:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767329.802846:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775767329.802856:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0a80 x1862022114681856/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767329.802863:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767329.802866:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775767329.802867:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767329.802870:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4707
00000100:00080000:2.0:1775767329.802872:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767329.802875:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767329.802879:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775767329.802888:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1f80 x1862022114681984/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775767329.802892:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114681728:192.168.202.110@tcp:38:kworker.0
00000100:00080000:2.0:1775767329.802900:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767329.802902:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1775767329.802903:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767329.802906:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4707
00000100:00080000:2.0:1775767329.802908:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767329.802910:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767329.802914:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775767329.802924:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2680 x1862022114682112/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775767329.802928:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114681856:192.168.202.110@tcp:38:kworker.0
00000100:00080000:2.0:1775767329.802939:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767329.802942:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775767329.802944:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767329.802948:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4707
00000100:00080000:2.0:1775767329.802952:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1775767329.802955:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114681984:192.168.202.110@tcp:8:kworker.0
00000100:00080000:2.0:1775767329.802959:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767329.802964:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775767329.802973:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2a00 x1862022114682240/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775767329.802980:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114682112:192.168.202.110@tcp:8:kworker.0
00000100:00080000:2.0:1775767329.802981:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767329.802988:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775767329.802989:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767329.802992:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4707
00000100:00080000:2.0:1775767329.802996:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767329.802998:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767329.803002:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767329.803007:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114682240:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767329.803014:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1180 x1862022114682368/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767329.803026:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1775767329.803028:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114682368:192.168.202.110@tcp:38:kworker.0
00000100:00080000:2.0:1775767329.803035:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775767329.803038:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767329.803044:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4707
00000100:00080000:2.0:1775767329.803049:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767329.803053:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767329.803059:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775767329.803072:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2d80 x1862022114682496/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775767329.803110:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114682496:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775767330.058772:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767330.826641:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767331.082739:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767331.850734:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767332.106734:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767332.874805:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000400:00000100:2.0:1775767332.874936:0:2392:0:(lib-socket.c:454:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.202.110:988
00000400:00000100:2.0:1775767332.874974:0:2392:0:(acceptor.c:111:lnet_connect_console_error()) Connection to 192.168.202.110@tcp at host 192.168.202.110 was unreachable: the network or that node may be down, or Lustre may be misconfigured.
00000800:00000100:2.0:1775767332.874989:0:2392:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:2.0:1775767332.875020:0:2392:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:2.0:1775767332.875028:0:2392:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:2.0:1775767332.875046:0:2392:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:2.0:1775767332.875058:0:2392:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:2.0:1775767332.875068:0:2392:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:2.0:1775767332.875074:0:2392:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000100:00100000:0.0:1775767332.875080:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767329/real 1775767332]  req@ffff9f14852d1f80 x1862022114681984/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767354 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000800:00000100:2.0:1775767332.875092:0:2392:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000100:00100000:0.0:1775767332.875118:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1f80 x1862022114681984/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767354 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767332.875132:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767332.875139:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767332.875147:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114681984:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775767332.875162:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767329/real 1775767332]  req@ffff9f14852d2680 x1862022114682112/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767354 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767332.875175:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2680 x1862022114682112/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767354 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767332.875185:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767332.875190:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767332.875194:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114682112:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775767332.875209:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767329/real 1775767332]  req@ffff9f14852d0700 x1862022114681728/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767354 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767332.875223:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0700 x1862022114681728/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767354 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767332.875231:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767332.875234:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767332.875237:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114681728:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767332.875244:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767329/real 1775767332]  req@ffff9f14852d0a80 x1862022114681856/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767354 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767332.875255:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0a80 x1862022114681856/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767354 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767332.875262:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767332.875266:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767332.875269:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114681856:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767332.875277:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767329/real 1775767332]  req@ffff9f14852d3800 x1862022114681600/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767354 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767332.875290:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3800 x1862022114681600/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767354 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767332.875297:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767332.875301:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767332.875304:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114681600:192.168.202.110@tcp:250:kworker.0
00000100:00100000:0.0:1775767332.875337:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767329/real 1775767332]  req@ffff9f14852d2d80 x1862022114682496/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767354 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767332.875367:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2d80 x1862022114682496/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767354 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767332.875376:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767332.875381:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767332.875384:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114682496:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775767332.875392:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767329/real 1775767332]  req@ffff9f14852d1180 x1862022114682368/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767354 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767332.875406:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1180 x1862022114682368/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767354 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767332.875415:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767332.875418:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767332.875422:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114682368:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767332.875431:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767329/real 1775767332]  req@ffff9f14852d2a00 x1862022114682240/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767354 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767332.875446:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2a00 x1862022114682240/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767354 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767332.875454:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767332.875457:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767332.875459:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114682240:192.168.202.110@tcp:38:kworker.0
00010000:00010000:2.0:1775767333.130790:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767333.898752:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767334.154789:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1775767334.922665:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767334.922675:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1775767334.922679:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767334.922699:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4712
00000100:00080000:0.0:1775767334.922709:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767334.922717:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775767334.922725:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:0.0:1775767334.922752:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d2a00 x1862022114682624/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767334.922782:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767334.922785:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775767334.922787:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767334.922790:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4712
00000100:00080000:0.0:1775767334.922794:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767334.922798:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767334.922808:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767334.922824:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3800 x1862022114682752/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767334.922836:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767334.922840:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:0.0:1775767334.922842:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767334.922847:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4712
00000100:00080000:0.0:1775767334.922852:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00100000:1.0:1775767334.922857:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114682624:192.168.202.110@tcp:250:kworker.0
00000100:00080000:0.0:1775767334.922861:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767334.922870:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767334.922883:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d1500 x1862022114682880/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767334.922895:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767334.922899:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775767334.922901:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767334.922905:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4712
00000100:00080000:0.0:1775767334.922914:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767334.922918:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767334.922924:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:1.0:1775767334.922924:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767334.922941:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0000 x1862022114683008/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767334.922951:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767334.922954:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1775767334.922956:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767334.922962:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4712
00000100:00100000:1.0:1775767334.922970:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767334/real 1775767334]  req@ffff9f14a04d2a00 x1862022114682624/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767359 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767334.922973:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767334.922981:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767334.922991:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775767334.923006:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d2300 x1862022114683136/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:1.0:1775767334.923016:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d2a00 x1862022114682624/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767359 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767334.923017:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767334.923025:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775767334.923029:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767334.923038:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4712
00000100:00080000:1.0:1775767334.923042:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767334.923043:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767334.923052:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:0.0:1775767334.923053:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767334.923062:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767334.923063:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114682624:192.168.202.110@tcp:250:kworker.0
00000100:00100000:0.0:1775767334.923078:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3100 x1862022114683264/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:1.0:1775767334.923078:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114682752:192.168.202.110@tcp:38:kworker.0
00000100:00080000:0.0:1775767334.923094:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767334.923098:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:0.0:1775767334.923100:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767334.923105:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4712
00000100:00080000:0.0:1775767334.923111:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767334.923118:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000800:00000100:1.0:1775767334.923118:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000008:00100000:0.0:1775767334.923127:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767334.923142:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767334/real 1775767334]  req@ffff9f14a04d3800 x1862022114682752/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767359 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767334.923144:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0a80 x1862022114683392/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767334.923173:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767334.923176:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775767334.923178:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:1.0:1775767334.923183:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3800 x1862022114682752/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767359 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767334.923184:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4712
00000100:00080000:0.0:1775767334.923192:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767334.923201:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00080000:1.0:1775767334.923203:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000008:00100000:0.0:1775767334.923207:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00080000:1.0:1775767334.923210:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767334.923217:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114682752:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767334.923226:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0e00 x1862022114683520/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:1.0:1775767334.923250:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114682880:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775767334.923275:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767334.923287:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767334/real 1775767334]  req@ffff9f14a04d1500 x1862022114682880/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767359 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767334.923298:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1500 x1862022114682880/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767359 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767334.923306:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767334.923309:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767334.923313:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114682880:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775767334.923320:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114683008:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775767334.923340:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767334.923350:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767334/real 1775767334]  req@ffff9f14a04d0000 x1862022114683008/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767359 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767334.923361:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0000 x1862022114683008/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767359 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767334.923369:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767334.923372:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767334.923375:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114683008:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775767334.923381:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114683136:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775767334.923398:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767334.923408:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767334/real 1775767334]  req@ffff9f14a04d2300 x1862022114683136/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767359 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767334.923419:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d2300 x1862022114683136/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767359 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767334.923426:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767334.923428:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767334.923431:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114683136:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775767334.923436:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114683264:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775767334.923454:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767334.923464:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767334/real 1775767334]  req@ffff9f14a04d3100 x1862022114683264/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767359 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767334.923476:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3100 x1862022114683264/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767359 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767334.923483:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767334.923486:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767334.923490:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114683264:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775767334.923495:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114683392:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775767334.923514:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767334.923595:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767334/real 1775767334]  req@ffff9f14a04d0a80 x1862022114683392/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767359 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767334.923610:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0a80 x1862022114683392/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767359 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767334.923619:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767334.923622:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767334.923625:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114683392:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775767334.923631:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114683520:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775767334.923655:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767334.923667:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767334/real 1775767334]  req@ffff9f14a04d0e00 x1862022114683520/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767359 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767334.923677:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0e00 x1862022114683520/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767359 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767334.923685:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767334.923689:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767334.923693:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114683520:192.168.202.110@tcp:8:kworker.0
00010000:00010000:0.0:1775767335.178722:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767335.946764:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767336.202671:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767336.970678:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767337.226762:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767337.994663:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767338.250739:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767339.018701:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767339.274736:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1775767340.042660:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:2.0:1775767340.042665:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:0.0:1775767340.042672:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1775767340.042678:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767340.042689:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4717
00000100:00080000:0.0:1775767340.042701:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767340.042707:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775767340.042740:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3b80 x1862022114683648/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767340.042763:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767340.042766:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775767340.042767:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767340.042770:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4717
00000100:00080000:0.0:1775767340.042773:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767340.042777:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767340.042784:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767340.042791:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d1180 x1862022114683776/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767340.042798:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767340.042800:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:0.0:1775767340.042801:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767340.042804:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4717
00000100:00080000:0.0:1775767340.042805:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767340.042807:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767340.042810:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767340.042816:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3480 x1862022114683904/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767340.042822:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767340.042824:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775767340.042826:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767340.042829:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4717
00000100:00080000:0.0:1775767340.042831:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767340.042834:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767340.042837:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775767340.042841:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114683648:192.168.202.110@tcp:250:kworker.0
00000100:00100000:0.0:1775767340.042846:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0380 x1862022114684032/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767340.042853:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767340.042855:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1775767340.042856:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767340.042859:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4717
00000100:00080000:0.0:1775767340.042861:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767340.042863:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767340.042866:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775767340.042874:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d1f80 x1862022114684160/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767340.042880:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767340.042883:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775767340.042883:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767340.042885:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4717
00000100:00080000:0.0:1775767340.042894:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767340.042897:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767340.042900:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000800:00000100:1.0:1775767340.042902:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767340.042915:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0e00 x1862022114684288/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767340.042922:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767340.042924:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:0.0:1775767340.042926:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767340.042928:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4717
00000100:00080000:0.0:1775767340.042931:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767340.042933:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767340.042937:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767340.042941:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767340/real 1775767340]  req@ffff9f14a04d3b80 x1862022114683648/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767365 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767340.042947:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0a80 x1862022114684416/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767340.042957:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767340.042959:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775767340.042963:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:1.0:1775767340.042963:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3b80 x1862022114683648/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767365 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767340.042972:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4717
00000100:00080000:0.0:1775767340.042979:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767340.042981:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767340.042982:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767340.042988:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00080000:1.0:1775767340.042989:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767340.042994:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114683648:192.168.202.110@tcp:250:kworker.0
00000100:00100000:0.0:1775767340.042998:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3100 x1862022114684544/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:1.0:1775767340.043005:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114683776:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775767340.043021:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767340.043041:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767340/real 1775767340]  req@ffff9f14a04d1180 x1862022114683776/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767365 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767340.043055:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1180 x1862022114683776/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767365 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767340.043061:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767340.043064:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767340.043066:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114683776:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775767340.043070:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114683904:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775767340.043084:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767340.043093:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767340/real 1775767340]  req@ffff9f14a04d3480 x1862022114683904/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767365 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767340.043102:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3480 x1862022114683904/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767365 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767340.043107:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767340.043110:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767340.043112:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114683904:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775767340.043132:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114684032:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775767340.043148:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767340.043156:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767340/real 1775767340]  req@ffff9f14a04d0380 x1862022114684032/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767365 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767340.043165:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0380 x1862022114684032/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767365 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767340.043170:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767340.043173:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767340.043176:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114684032:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775767340.043180:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114684160:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775767340.043193:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767340.043201:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767340/real 1775767340]  req@ffff9f14a04d1f80 x1862022114684160/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767365 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767340.043215:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1f80 x1862022114684160/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767365 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767340.043223:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767340.043226:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767340.043230:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114684160:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775767340.043237:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114684288:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775767340.043259:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767340.043273:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767340/real 1775767340]  req@ffff9f14a04d0e00 x1862022114684288/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767365 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767340.043287:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0e00 x1862022114684288/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767365 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767340.043306:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767340.043311:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767340.043315:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114684288:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775767340.043322:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114684416:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775767340.043345:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767340.043356:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767340/real 1775767340]  req@ffff9f14a04d0a80 x1862022114684416/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767365 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767340.043365:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0a80 x1862022114684416/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767365 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767340.043371:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767340.043374:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767340.043379:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114684416:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775767340.043388:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114684544:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775767340.043413:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767340.043425:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767340/real 1775767340]  req@ffff9f14a04d3100 x1862022114684544/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767365 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767340.043439:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3100 x1862022114684544/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767365 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767340.043447:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767340.043449:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767340.043452:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114684544:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775767340.298716:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767341.066657:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767341.322722:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767342.090704:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:0.0:1775767342.346798:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767343.114663:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767343.370674:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767344.138645:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767344.394715:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767345.162684:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:1.0:1775767345.162709:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767345.162718:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1775767345.162723:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767345.162734:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4722
00000100:00080000:1.0:1775767345.162746:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767345.162753:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775767345.162801:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff800 x1862022114684672/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767345.162829:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767345.162832:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775767345.162834:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767345.162837:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4722
00000100:00080000:1.0:1775767345.162840:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767345.162842:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767345.162854:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767345.162868:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fed80 x1862022114684800/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767345.162879:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767345.162883:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775767345.162885:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775767345.162892:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114684672:192.168.202.110@tcp:250:kworker.0
00000100:00080000:1.0:1775767345.162894:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4722
00000100:00080000:1.0:1775767345.162904:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767345.162910:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767345.162916:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767345.162930:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ffb80 x1862022114684928/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767345.162938:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767345.162941:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775767345.162942:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767345.162945:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4722
00000100:00080000:1.0:1775767345.162970:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767345.162973:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767345.162977:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:0.0:1775767345.162983:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767345.162988:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fca80 x1862022114685056/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767345.162996:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767345.162998:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1775767345.163000:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767345.163003:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4722
00000100:00080000:1.0:1775767345.163006:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767345.163008:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767345.163019:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775767345.163032:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fea00 x1862022114685184/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767345.163039:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767345.163042:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775767345.163046:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775767345.163047:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767345/real 1775767345]  req@ffff9f14ab4ff800 x1862022114684672/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767370 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767345.163054:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4722
00000100:00080000:1.0:1775767345.163061:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767345.163066:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767345.163070:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767345.163075:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff800 x1862022114684672/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767370 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767345.163080:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fce00 x1862022114685312/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767345.163090:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767345.163093:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767345.163097:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:1.0:1775767345.163098:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775767345.163101:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775767345.163104:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114684672:192.168.202.110@tcp:250:kworker.0
00000100:00080000:1.0:1775767345.163108:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4722
00000100:00080000:1.0:1775767345.163114:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767345.163128:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767345.163133:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767345.163144:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114684800:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775767345.163148:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdf80 x1862022114685440/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767345.163163:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000800:00000100:0.0:1775767345.163168:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:1.0:1775767345.163171:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775767345.163175:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775767345.163181:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767345/real 1775767345]  req@ffff9f14ab4fed80 x1862022114684800/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767370 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767345.163182:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4722
00000100:00080000:1.0:1775767345.163191:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1775767345.163197:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fed80 x1862022114684800/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767370 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767345.163199:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767345.163210:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00080000:0.0:1775767345.163212:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767345.163219:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767345.163224:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114684800:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775767345.163226:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdc00 x1862022114685568/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775767345.163237:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ffb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114684928:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767345.163264:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767345.163276:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767345/real 1775767345]  req@ffff9f14ab4ffb80 x1862022114684928/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767370 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767345.163284:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ffb80 x1862022114684928/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767370 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767345.163290:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767345.163292:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767345.163304:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ffb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114684928:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767345.163311:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114685056:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767345.163334:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767345.163345:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767345/real 1775767345]  req@ffff9f14ab4fca80 x1862022114685056/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767370 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767345.163363:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fca80 x1862022114685056/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767370 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767345.163369:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767345.163371:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767345.163373:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114685056:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775767345.163377:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114685184:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767345.163400:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767345.163420:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767345/real 1775767345]  req@ffff9f14ab4fea00 x1862022114685184/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767370 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767345.163449:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fea00 x1862022114685184/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767370 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767345.163463:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767345.163465:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767345.163467:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114685184:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775767345.163471:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114685312:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767345.163488:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767345.163496:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767345/real 1775767345]  req@ffff9f14ab4fce00 x1862022114685312/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767370 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767345.163503:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fce00 x1862022114685312/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767370 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767345.163523:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767345.163525:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767345.163527:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114685312:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767345.163541:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114685440:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767345.163552:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767345.163580:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767345/real 1775767345]  req@ffff9f14ab4fdf80 x1862022114685440/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767370 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767345.163588:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdf80 x1862022114685440/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767370 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767345.163596:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767345.163600:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767345.163603:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114685440:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767345.163609:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114685568:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767345.163636:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767345.163648:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767345/real 1775767345]  req@ffff9f14ab4fdc00 x1862022114685568/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767370 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767345.163661:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdc00 x1862022114685568/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767370 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767345.163667:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767345.163669:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767345.163672:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114685568:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775767345.418719:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767346.186694:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767346.442723:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767347.210638:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767347.466676:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767348.234696:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:0.0:1775767348.490786:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767349.258657:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:0.0:1775767349.514658:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1775767350.282641:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:2.0:1775767350.282641:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:1.0:1775767350.282650:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1775767350.282654:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767350.282662:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4727
00000100:00080000:1.0:1775767350.282674:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767350.282683:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775767350.282732:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff100 x1862022114685696/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767350.282754:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767350.282756:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775767350.282758:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767350.282761:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4727
00000100:00080000:1.0:1775767350.282765:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767350.282769:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767350.282777:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767350.282779:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114685696:192.168.202.110@tcp:250:kworker.0
00000100:00100000:1.0:1775767350.282791:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fe300 x1862022114685824/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767350.282799:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767350.282801:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775767350.282803:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767350.282806:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4727
00000100:00080000:1.0:1775767350.282810:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767350.282813:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767350.282816:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767350.282823:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fc380 x1862022114685952/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767350.282829:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767350.282832:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000800:00000100:0.0:1775767350.282835:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:1.0:1775767350.282836:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767350.282843:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4727
00000100:00080000:1.0:1775767350.282846:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767350.282849:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767350.282853:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775767350.282862:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fd880 x1862022114686080/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767350.282869:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767350.282873:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00100000:0.0:1775767350.282874:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767350/real 1775767350]  req@ffff9f14ab4ff100 x1862022114685696/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767375 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767350.282875:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767350.282882:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4727
00000100:00080000:1.0:1775767350.282887:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767350.282892:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767350.282896:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775767350.282899:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff100 x1862022114685696/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767375 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767350.282906:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdc00 x1862022114686208/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767350.282908:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767350.282916:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767350.282922:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114685696:192.168.202.110@tcp:250:kworker.0
00000100:00080000:1.0:1775767350.282934:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767350.282937:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775767350.282938:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767350.282941:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4727
00000100:00080000:1.0:1775767350.282943:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767350.282946:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767350.282948:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767350.282956:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdf80 x1862022114686336/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775767350.282958:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114685824:192.168.202.110@tcp:38:kworker.0
00000100:00080000:1.0:1775767350.282972:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767350.282975:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775767350.282977:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767350.282980:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4727
00000100:00080000:1.0:1775767350.282983:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000800:00000100:0.0:1775767350.282985:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:1.0:1775767350.282989:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767350.282993:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767350.282999:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767350/real 1775767350]  req@ffff9f14ab4fe300 x1862022114685824/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767375 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767350.283004:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fce00 x1862022114686464/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767350.283019:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1775767350.283021:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fe300 x1862022114685824/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767375 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767350.283028:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775767350.283032:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767350.283041:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767350.283041:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4727
00000100:00080000:0.0:1775767350.283048:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00080000:1.0:1775767350.283048:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1775767350.283055:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114685824:192.168.202.110@tcp:38:kworker.0
00000100:00080000:1.0:1775767350.283055:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767350.283059:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775767350.283067:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fea00 x1862022114686592/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775767350.283072:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114685952:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767350.283095:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767350.283104:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767350/real 1775767350]  req@ffff9f14ab4fc380 x1862022114685952/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767375 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767350.283113:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fc380 x1862022114685952/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767375 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767350.283118:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767350.283121:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767350.283123:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114685952:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767350.283126:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114686080:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767350.283137:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767350.283142:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767350/real 1775767350]  req@ffff9f14ab4fd880 x1862022114686080/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767375 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767350.283149:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fd880 x1862022114686080/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767375 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767350.283155:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767350.283157:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767350.283158:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114686080:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775767350.283162:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114686208:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767350.283172:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767350.283181:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767350/real 1775767350]  req@ffff9f14ab4fdc00 x1862022114686208/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767375 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767350.283202:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdc00 x1862022114686208/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767375 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767350.283211:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767350.283214:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767350.283218:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114686208:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775767350.283235:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114686336:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767350.283252:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767350.283259:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767350/real 1775767350]  req@ffff9f14ab4fdf80 x1862022114686336/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767375 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767350.283267:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdf80 x1862022114686336/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767375 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767350.283273:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767350.283275:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767350.283277:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114686336:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767350.283281:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114686464:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767350.283290:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767350.283296:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767350/real 1775767350]  req@ffff9f14ab4fce00 x1862022114686464/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767375 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767350.283303:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fce00 x1862022114686464/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767375 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767350.283313:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767350.283315:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767350.283317:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114686464:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767350.283320:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114686592:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767350.283329:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767350.283334:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767350/real 1775767350]  req@ffff9f14ab4fea00 x1862022114686592/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767375 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767350.283340:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fea00 x1862022114686592/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767375 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767350.283345:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767350.283347:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767350.283349:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114686592:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775767350.538718:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767351.306622:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767351.562703:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767352.330663:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767352.586794:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767353.354651:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767353.610699:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767354.378625:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767354.634727:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767355.402692:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775767355.402700:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767355.402710:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775767355.402715:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767355.402730:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4732
00000100:00080000:3.0:1775767355.402745:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767355.402753:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775767355.402811:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747b80 x1862022114686720/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767355.402843:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767355.402848:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775767355.402850:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767355.402855:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4732
00000100:00080000:3.0:1775767355.402859:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767355.402864:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767355.402876:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767355.402893:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746300 x1862022114686848/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767355.402914:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767355.402919:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775767355.402923:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775767355.402924:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114686720:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775767355.402939:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4732
00000100:00080000:3.0:1775767355.402944:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767355.402948:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767355.402954:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767355.402972:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745880 x1862022114686976/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767355.402984:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767355.402988:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775767355.402990:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767355.402995:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4732
00000100:00080000:3.0:1775767355.403000:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000800:00000100:0.0:1775767355.403010:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775767355.403010:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767355.403021:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775767355.403037:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747480 x1862022114687104/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767355.403050:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1775767355.403054:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767355/real 1775767355]  req@ffff9f1485747b80 x1862022114686720/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767380 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767355.403065:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775767355.403070:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767355.403086:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4732
00000100:00100000:0.0:1775767355.403092:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747b80 x1862022114686720/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767380 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767355.403095:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767355.403108:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00080000:0.0:1775767355.403110:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767355.403117:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000008:00100000:3.0:1775767355.403117:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775767355.403127:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114686720:192.168.202.110@tcp:250:kworker.0
00000100:00100000:3.0:1775767355.403143:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744a80 x1862022114687232/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775767355.403152:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114686848:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775767355.403174:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767355.403178:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775767355.403182:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000800:00000100:0.0:1775767355.403183:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775767355.403194:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4732
00000100:00100000:0.0:1775767355.403200:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767355/real 1775767355]  req@ffff9f1485746300 x1862022114686848/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767380 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767355.403202:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767355.403214:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775767355.403217:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746300 x1862022114686848/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767380 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:3.0:1775767355.403231:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00080000:0.0:1775767355.403232:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767355.403241:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767355.403244:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114686848:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767355.403249:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114686976:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767355.403256:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746a00 x1862022114687360/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1775767355.403266:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767355.403280:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767355/real 1775767355]  req@ffff9f1485745880 x1862022114686976/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767380 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767355.403285:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767355.403290:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775767355.403293:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767355.403310:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4732
00000100:00100000:0.0:1775767355.403317:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745880 x1862022114686976/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767380 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767355.403324:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767355.403326:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767355.403344:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767355.403350:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114686976:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775767355.403350:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775767355.403370:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114687104:192.168.202.110@tcp:8:kworker.0
00000008:00100000:3.0:1775767355.403371:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767355.403402:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744380 x1862022114687488/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1775767355.403411:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767355.403428:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767355/real 1775767355]  req@ffff9f1485747480 x1862022114687104/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767380 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767355.403474:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747480 x1862022114687104/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767380 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767355.403475:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767355.403491:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767355.403493:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767355.403497:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114687104:192.168.202.110@tcp:8:kworker.0
00000100:00080000:3.0:1775767355.403501:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775767355.403505:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767355.403512:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4732
00000100:00100000:0.0:1775767355.403517:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114687232:192.168.202.110@tcp:8:kworker.0
00000100:00080000:3.0:1775767355.403524:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767355.403532:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000800:00000100:0.0:1775767355.403540:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000008:00100000:3.0:1775767355.403543:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775767355.403593:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767355/real 1775767355]  req@ffff9f1485744a80 x1862022114687232/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767380 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767355.403602:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744e00 x1862022114687616/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775767355.403631:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744a80 x1862022114687232/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767380 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767355.403639:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767355.403645:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767355.403649:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114687232:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775767355.403654:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114687360:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767355.403673:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767355.403681:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767355/real 1775767355]  req@ffff9f1485746a00 x1862022114687360/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767380 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767355.403689:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746a00 x1862022114687360/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767380 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767355.403698:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767355.403700:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767355.403702:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114687360:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767355.403706:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114687488:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767355.403715:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767355.403721:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767355/real 1775767355]  req@ffff9f1485744380 x1862022114687488/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767380 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767355.403727:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744380 x1862022114687488/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767380 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767355.403732:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767355.403734:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767355.403736:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114687488:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767355.403746:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114687616:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767355.403756:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767355.403761:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767355/real 1775767355]  req@ffff9f1485744e00 x1862022114687616/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767380 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767355.403769:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744e00 x1862022114687616/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767380 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767355.403773:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767355.403775:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767355.403778:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114687616:192.168.202.110@tcp:8:kworker.0
00010000:00010000:0.0:1775767355.658680:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767356.426609:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:0.0:1775767356.682691:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767357.450614:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767357.706742:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767358.474634:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767358.730701:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767359.498657:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767359.754668:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767360.522617:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775767360.522617:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767360.522626:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775767360.522630:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767360.522638:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4738
00000100:00080000:3.0:1775767360.522647:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767360.522655:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775767360.522694:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745500 x1862022114687744/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767360.522730:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767360.522733:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775767360.522735:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767360.522738:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4738
00000100:00080000:3.0:1775767360.522741:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767360.522744:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767360.522754:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767360.522765:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745f80 x1862022114687872/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775767360.522773:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114687744:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775767360.522776:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767360.522780:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775767360.522782:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767360.522785:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4738
00000100:00080000:3.0:1775767360.522787:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767360.522789:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767360.522792:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767360.522800:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747800 x1862022114688000/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767360.522807:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767360.522809:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775767360.522811:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767360.522813:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4738
00000100:00080000:3.0:1775767360.522815:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767360.522817:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767360.522821:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775767360.522828:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744000 x1862022114688128/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1775767360.522834:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775767360.522840:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767360.522843:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775767360.522844:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767360.522848:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4738
00000100:00080000:3.0:1775767360.522852:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767360.522855:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767360.522858:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775767360.522865:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746680 x1862022114688256/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767360.522871:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767360.522873:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775767360.522875:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767360.522878:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4738
00000100:00080000:3.0:1775767360.522881:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1775767360.522882:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767360/real 1775767360]  req@ffff9f1485745500 x1862022114687744/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767385 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767360.522888:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767360.522895:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767360.522906:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745c00 x1862022114688384/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775767360.522908:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745500 x1862022114687744/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767385 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767360.522929:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767360.522931:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767360.522932:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775767360.522936:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767360.522939:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:3.0:1775767360.522943:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4738
00000100:00100000:0.0:1775767360.522945:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114687744:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775767360.522946:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767360.522955:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767360.522960:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767360.522970:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114687872:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767360.522975:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745180 x1862022114688512/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767360.522988:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000800:00000100:0.0:1775767360.522989:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775767360.522995:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775767360.522998:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775767360.523001:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767360/real 1775767360]  req@ffff9f1485745f80 x1862022114687872/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767385 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767360.523005:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4738
00000100:00080000:3.0:1775767360.523013:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1775767360.523018:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745f80 x1862022114687872/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767385 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767360.523019:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767360.523028:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00080000:0.0:1775767360.523030:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767360.523035:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767360.523037:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114687872:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767360.523042:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747100 x1862022114688640/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775767360.523044:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114688000:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767360.523068:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767360.523082:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767360/real 1775767360]  req@ffff9f1485747800 x1862022114688000/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767385 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767360.523095:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747800 x1862022114688000/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767385 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767360.523104:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767360.523108:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767360.523111:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114688000:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767360.523118:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114688128:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767360.523136:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767360.523143:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767360/real 1775767360]  req@ffff9f1485744000 x1862022114688128/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767385 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767360.523151:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744000 x1862022114688128/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767385 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767360.523157:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767360.523159:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767360.523161:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114688128:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775767360.523164:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114688256:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767360.523174:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767360.523179:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767360/real 1775767360]  req@ffff9f1485746680 x1862022114688256/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767385 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767360.523191:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746680 x1862022114688256/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767385 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767360.523195:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767360.523197:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767360.523199:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114688256:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775767360.523202:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114688384:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767360.523211:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767360.523217:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767360/real 1775767360]  req@ffff9f1485745c00 x1862022114688384/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767385 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767360.523224:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745c00 x1862022114688384/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767385 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767360.523229:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767360.523231:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767360.523233:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114688384:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767360.523244:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114688512:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767360.523254:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767360.523259:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767360/real 1775767360]  req@ffff9f1485745180 x1862022114688512/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767385 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767360.523266:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745180 x1862022114688512/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767385 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767360.523278:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767360.523313:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767360.523318:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114688512:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767360.523325:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114688640:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767360.523345:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767360.523353:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767360/real 1775767360]  req@ffff9f1485747100 x1862022114688640/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767385 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767360.523361:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747100 x1862022114688640/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767385 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767360.523367:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767360.523369:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767360.523371:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114688640:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775767360.778666:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767361.546658:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767361.802748:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767362.570672:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:0.0:1775767362.826660:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767363.594689:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:0.0:1775767363.850694:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767364.618664:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767364.874695:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1775767365.642653:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767365.642662:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1775767365.642665:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00100000:3.0:1775767365.642668:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:2.0:1775767365.642674:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4743
00000100:00080000:2.0:1775767365.642683:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767365.642689:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775767365.642721:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1500 x1862022114688768/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767365.642745:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767365.642748:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775767365.642749:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767365.642752:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4743
00000100:00080000:2.0:1775767365.642754:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767365.642757:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767365.642764:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775767365.642772:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0380 x1862022114688896/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767365.642780:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767365.642783:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775767365.642784:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767365.642787:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4743
00000100:00080000:2.0:1775767365.642790:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767365.642792:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767365.642795:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775767365.642804:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0000 x1862022114689024/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775767365.642805:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114688768:192.168.202.110@tcp:250:kworker.0
00000100:00080000:2.0:1775767365.642819:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767365.642821:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775767365.642822:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767365.642825:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4743
00000100:00080000:2.0:1775767365.642827:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767365.642829:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767365.642832:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775767365.642839:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3b80 x1862022114689152/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767365.642845:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767365.642847:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1775767365.642848:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767365.642850:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4743
00000100:00080000:2.0:1775767365.642853:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767365.642855:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767365.642860:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775767365.642870:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0e00 x1862022114689280/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1775767365.642871:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:2.0:1775767365.642879:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767365.642882:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775767365.642883:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767365.642887:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4743
00000100:00080000:2.0:1775767365.642890:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767365.642892:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767365.642895:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775767365.642902:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1880 x1862022114689408/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767365.642908:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1775767365.642912:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767365/real 1775767365]  req@ffff9f14852d1500 x1862022114688768/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767390 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767365.642912:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775767365.642915:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767365.642924:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4743
00000100:00080000:2.0:1775767365.642929:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767365.642931:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767365.642935:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767365.642940:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1500 x1862022114688768/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767390 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767365.642944:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3100 x1862022114689536/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767365.642966:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767365.642966:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767365.642972:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775767365.642974:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767365.642976:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:2.0:1775767365.642980:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4743
00000100:00100000:0.0:1775767365.642984:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114688768:192.168.202.110@tcp:250:kworker.0
00000100:00080000:2.0:1775767365.642984:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767365.642991:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767365.642995:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775767365.642997:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114688896:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767365.643004:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2300 x1862022114689664/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1775767365.643019:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767365.643032:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767365/real 1775767365]  req@ffff9f14852d0380 x1862022114688896/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767390 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767365.643041:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0380 x1862022114688896/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767390 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767365.643046:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767365.643049:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767365.643051:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114688896:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767365.643069:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114689024:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767365.643080:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767365.643085:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767365/real 1775767365]  req@ffff9f14852d0000 x1862022114689024/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767390 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767365.643092:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0000 x1862022114689024/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767390 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767365.643097:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767365.643099:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767365.643101:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114689024:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767365.643105:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114689152:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767365.643115:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767365.643121:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767365/real 1775767365]  req@ffff9f14852d3b80 x1862022114689152/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767390 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767365.643128:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3b80 x1862022114689152/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767390 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767365.643133:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767365.643135:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767365.643137:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114689152:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775767365.643140:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114689280:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767365.643148:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767365.643163:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767365/real 1775767365]  req@ffff9f14852d0e00 x1862022114689280/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767390 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767365.643171:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0e00 x1862022114689280/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767390 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767365.643176:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767365.643179:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767365.643181:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114689280:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775767365.643184:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114689408:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767365.643195:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767365.643201:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767365/real 1775767365]  req@ffff9f14852d1880 x1862022114689408/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767390 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767365.643213:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1880 x1862022114689408/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767390 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767365.643218:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767365.643219:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767365.643221:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114689408:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767365.643224:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114689536:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767365.643246:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767365.643254:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767365/real 1775767365]  req@ffff9f14852d3100 x1862022114689536/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767390 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767365.643263:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3100 x1862022114689536/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767390 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767365.643268:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767365.643270:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767365.643273:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114689536:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767365.643276:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114689664:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767365.643287:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767365.643293:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767365/real 1775767365]  req@ffff9f14852d2300 x1862022114689664/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767390 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767365.643300:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2300 x1862022114689664/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767390 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767365.643310:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767365.643312:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767365.643314:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114689664:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775767365.898738:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775767366.666633:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767366.922746:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775767367.690682:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767367.946807:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775767368.714676:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767368.970758:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775767369.738740:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:0.0:1775767369.994652:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775767370.762645:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:1.0:1775767370.762665:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767370.762672:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1775767370.762676:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767370.762686:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4748
00000100:00080000:1.0:1775767370.762697:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767370.762705:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775767370.762741:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fca80 x1862022114689792/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767370.762770:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767370.762773:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775767370.762775:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767370.762779:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4748
00000100:00080000:1.0:1775767370.762783:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767370.762786:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767370.762796:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767370.762810:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ffb80 x1862022114689920/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767370.762822:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1775767370.762824:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114689792:192.168.202.110@tcp:250:kworker.0
00000100:00080000:1.0:1775767370.762833:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775767370.762836:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767370.762841:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4748
00000100:00080000:1.0:1775767370.762845:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767370.762848:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767370.762854:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767370.762866:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fed80 x1862022114690048/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767370.762876:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767370.762880:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775767370.762882:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767370.762886:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4748
00000100:00080000:1.0:1775767370.762890:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767370.762896:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000800:00000100:0.0:1775767370.762897:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000008:00100000:1.0:1775767370.762903:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775767370.762916:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff800 x1862022114690176/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767370.762925:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767370.762929:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1775767370.762930:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767370.762934:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4748
00000100:00080000:1.0:1775767370.762938:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767370.762944:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775767370.762945:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767370/real 1775767370]  req@ffff9f14ab4fca80 x1862022114689792/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767395 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:1.0:1775767370.762952:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775767370.762967:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fea00 x1862022114690304/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775767370.762975:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fca80 x1862022114689792/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767395 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767370.762982:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767370.762990:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775767370.762994:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767370.762999:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767370.763001:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4748
00000100:00080000:1.0:1775767370.763007:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767370.763009:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:1.0:1775767370.763014:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775767370.763019:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114689792:192.168.202.110@tcp:250:kworker.0
00000008:00100000:1.0:1775767370.763021:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767370.763036:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fce00 x1862022114690432/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767370.763048:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1775767370.763052:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ffb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114689920:192.168.202.110@tcp:38:kworker.0
00000100:00080000:1.0:1775767370.763055:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775767370.763059:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767370.763064:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4748
00000100:00080000:1.0:1775767370.763068:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767370.763072:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767370.763077:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767370.763089:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdf80 x1862022114690560/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1775767370.763090:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:1.0:1775767370.763102:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767370.763105:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775767370.763107:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775767370.763112:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767370/real 1775767370]  req@ffff9f14ab4ffb80 x1862022114689920/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767395 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767370.763114:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4748
00000100:00080000:1.0:1775767370.763120:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767370.763127:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767370.763135:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775767370.763140:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ffb80 x1862022114689920/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767395 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767370.763147:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdc00 x1862022114690688/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767370.763157:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767370.763165:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767370.763171:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ffb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114689920:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767370.763179:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114690048:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767370.763227:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767370.763240:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767370/real 1775767370]  req@ffff9f14ab4fed80 x1862022114690048/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767395 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767370.763254:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fed80 x1862022114690048/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767395 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767370.763263:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767370.763266:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767370.763271:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114690048:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767370.763277:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114690176:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767370.763298:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767370.763310:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767370/real 1775767370]  req@ffff9f14ab4ff800 x1862022114690176/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767395 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767370.763324:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff800 x1862022114690176/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767395 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767370.763332:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767370.763336:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767370.763339:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114690176:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775767370.763345:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114690304:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767370.763363:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767370.763373:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767370/real 1775767370]  req@ffff9f14ab4fea00 x1862022114690304/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767395 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767370.763386:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fea00 x1862022114690304/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767395 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767370.763394:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767370.763397:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767370.763401:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114690304:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775767370.763416:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114690432:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767370.763434:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767370.763445:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767370/real 1775767370]  req@ffff9f14ab4fce00 x1862022114690432/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767395 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767370.763456:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fce00 x1862022114690432/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767395 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767370.763465:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767370.763468:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767370.763472:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114690432:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767370.763477:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114690560:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767370.763494:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767370.763504:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767370/real 1775767370]  req@ffff9f14ab4fdf80 x1862022114690560/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767395 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767370.763516:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdf80 x1862022114690560/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767395 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767370.763524:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767370.763528:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767370.763531:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114690560:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767370.763537:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114690688:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767370.763579:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767370.763591:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767370/real 1775767370]  req@ffff9f14ab4fdc00 x1862022114690688/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767395 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767370.763604:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdc00 x1862022114690688/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767395 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767370.763613:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767370.763616:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767370.763620:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114690688:192.168.202.110@tcp:8:kworker.0
00010000:00010000:0.0:1775767371.018719:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775767371.786661:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767372.042728:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775767372.810651:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767373.066768:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775767373.834692:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767374.090720:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775767374.858656:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767375.114724:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1775767375.882706:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:3.0:1775767375.882706:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:1.0:1775767375.882716:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1775767375.882722:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767375.882741:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4753
00000100:00080000:1.0:1775767375.882755:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767375.882765:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775767375.882829:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fd880 x1862022114690816/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767375.882874:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767375.882877:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775767375.882879:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767375.882883:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4753
00000100:00080000:1.0:1775767375.882888:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767375.882892:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767375.882903:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767375.882920:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fc380 x1862022114690944/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767375.882953:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767375.882966:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775767375.882968:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767375.882973:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4753
00000100:00080000:1.0:1775767375.882978:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767375.882981:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767375.882985:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767375.882994:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fe300 x1862022114691072/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767375.883005:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767375.883008:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775767375.883010:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767375.883023:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4753
00000100:00080000:1.0:1775767375.883029:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767375.883036:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767375.883044:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775767375.883049:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114690816:192.168.202.110@tcp:250:kworker.0
00000100:00100000:1.0:1775767375.883063:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff100 x1862022114691200/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767375.883075:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767375.883079:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1775767375.883081:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767375.883086:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4753
00000100:00080000:1.0:1775767375.883092:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767375.883097:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767375.883104:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775767375.883116:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdc00 x1862022114691328/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767375.883124:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767375.883127:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775767375.883129:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767375.883132:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4753
00000100:00080000:1.0:1775767375.883135:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767375.883155:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767375.883159:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767375.883169:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdf80 x1862022114691456/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767375.883191:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767375.883193:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775767375.883194:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767375.883197:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4753
00000100:00080000:1.0:1775767375.883200:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767375.883202:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767375.883206:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767375.883213:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fce00 x1862022114691584/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767375.883224:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767375.883227:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775767375.883228:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767375.883230:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4753
00000100:00080000:1.0:1775767375.883233:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767375.883235:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767375.883237:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775767375.883243:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fea00 x1862022114691712/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:3.0:1775767375.883374:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767375.883421:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767375/real 1775767375]  req@ffff9f14ab4fd880 x1862022114690816/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767400 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767375.883451:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fd880 x1862022114690816/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767400 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767375.883463:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767375.883468:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767375.883472:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114690816:192.168.202.110@tcp:250:kworker.0
00000100:00100000:3.0:1775767375.883500:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114690944:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775767375.883533:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767375.883544:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767375/real 1775767375]  req@ffff9f14ab4fc380 x1862022114690944/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767400 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767375.883608:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fc380 x1862022114690944/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767400 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767375.883621:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767375.883626:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767375.883630:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114690944:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767375.883638:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114691072:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775767375.883680:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767375.883694:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767375/real 1775767375]  req@ffff9f14ab4fe300 x1862022114691072/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767400 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767375.883719:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fe300 x1862022114691072/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767400 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767375.883734:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767375.883737:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767375.883740:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114691072:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767375.883747:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114691200:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775767375.883769:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767375.883779:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767375/real 1775767375]  req@ffff9f14ab4ff100 x1862022114691200/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767400 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767375.883791:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff100 x1862022114691200/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767400 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767375.883800:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767375.883803:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767375.883806:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114691200:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775767375.883812:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114691328:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775767375.883833:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767375.883857:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767375/real 1775767375]  req@ffff9f14ab4fdc00 x1862022114691328/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767400 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767375.883868:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdc00 x1862022114691328/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767400 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767375.883876:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767375.883879:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767375.883882:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114691328:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775767375.883889:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114691456:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775767375.883922:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767375.883934:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767375/real 1775767375]  req@ffff9f14ab4fdf80 x1862022114691456/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767400 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767375.883949:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdf80 x1862022114691456/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767400 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767375.883958:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767375.883962:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767375.883966:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114691456:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767375.883973:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114691584:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775767375.883994:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767375.884005:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767375/real 1775767375]  req@ffff9f14ab4fce00 x1862022114691584/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767400 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767375.884020:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fce00 x1862022114691584/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767400 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767375.884030:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767375.884043:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767375.884047:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114691584:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767375.884055:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114691712:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775767375.884080:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767375.884093:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767375/real 1775767375]  req@ffff9f14ab4fea00 x1862022114691712/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767400 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767375.884118:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fea00 x1862022114691712/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767400 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767375.884127:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767375.884132:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767375.884136:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114691712:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775767376.138750:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775767376.906692:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:0.0:1775767377.162721:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775767377.930690:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:0.0:1775767378.186768:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775767378.954705:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767379.210798:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775767379.978704:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767380.234696:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1775767381.002670:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:3.0:1775767381.002670:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:2.0:1775767381.002683:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1775767381.002689:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767381.002701:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4758
00000100:00080000:2.0:1775767381.002716:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767381.002727:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775767381.002779:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1c00 x1862022114691840/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767381.002805:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767381.002809:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775767381.002811:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767381.002814:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4758
00000100:00080000:2.0:1775767381.002817:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767381.002821:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767381.002831:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775767381.002841:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3480 x1862022114691968/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775767381.002846:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114691840:192.168.202.110@tcp:250:kworker.0
00000100:00080000:2.0:1775767381.002855:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767381.002858:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775767381.002859:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767381.002862:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4758
00000100:00080000:2.0:1775767381.002864:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767381.002867:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767381.002869:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775767381.002879:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2300 x1862022114692096/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767381.002885:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767381.002888:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775767381.002890:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767381.002895:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4758
00000100:00080000:2.0:1775767381.002899:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767381.002904:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767381.002911:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775767381.002924:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3100 x1862022114692224/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767381.002935:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767381.002941:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000800:00000100:3.0:1775767381.002941:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:2.0:1775767381.002947:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767381.002952:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4758
00000100:00080000:2.0:1775767381.002957:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767381.002961:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767381.002967:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775767381.002979:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1880 x1862022114692352/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767381.002987:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767381.002990:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775767381.002991:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767381.002995:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4758
00000100:00080000:2.0:1775767381.003014:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:3.0:1775767381.003017:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767381/real 1775767381]  req@ffff9f14852d1c00 x1862022114691840/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767406 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767381.003021:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767381.003032:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775767381.003049:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0e00 x1862022114692480/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775767381.003052:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1c00 x1862022114691840/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767406 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767381.003079:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767381.003082:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767381.003083:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775767381.003087:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767381.003109:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:2.0:1775767381.003111:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4758
00000100:00080000:2.0:1775767381.003117:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:3.0:1775767381.003119:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114691840:192.168.202.110@tcp:250:kworker.0
00000100:00080000:2.0:1775767381.003123:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767381.003148:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775767381.003159:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3b80 x1862022114692608/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767381.003165:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767381.003168:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775767381.003170:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:3.0:1775767381.003174:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114691968:192.168.202.110@tcp:38:kworker.0
00000100:00080000:2.0:1775767381.003175:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4758
00000100:00080000:2.0:1775767381.003180:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767381.003185:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767381.003188:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775767381.003197:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0000 x1862022114692736/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:3.0:1775767381.003215:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767381.003233:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767381/real 1775767381]  req@ffff9f14852d3480 x1862022114691968/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767406 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767381.003248:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3480 x1862022114691968/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767406 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767381.003257:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767381.003260:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767381.003264:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114691968:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767381.003271:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114692096:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775767381.003292:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767381.003302:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767381/real 1775767381]  req@ffff9f14852d2300 x1862022114692096/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767406 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767381.003315:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2300 x1862022114692096/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767406 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767381.003324:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767381.003328:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767381.003332:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114692096:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767381.003338:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114692224:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775767381.003360:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767381.003370:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767381/real 1775767381]  req@ffff9f14852d3100 x1862022114692224/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767406 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767381.003383:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3100 x1862022114692224/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767406 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767381.003392:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767381.003396:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767381.003400:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114692224:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775767381.003406:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114692352:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775767381.003441:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767381.003452:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767381/real 1775767381]  req@ffff9f14852d1880 x1862022114692352/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767406 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767381.003465:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1880 x1862022114692352/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767406 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767381.003475:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767381.003478:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767381.003482:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114692352:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775767381.003503:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114692480:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775767381.003523:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767381.003533:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767381/real 1775767381]  req@ffff9f14852d0e00 x1862022114692480/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767406 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767381.003546:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0e00 x1862022114692480/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767406 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767381.003555:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767381.003558:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767381.003562:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114692480:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767381.003568:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114692608:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775767381.003588:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767381.003620:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767381/real 1775767381]  req@ffff9f14852d3b80 x1862022114692608/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767406 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767381.003634:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3b80 x1862022114692608/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767406 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767381.003642:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767381.003646:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767381.003649:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114692608:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767381.003673:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114692736:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775767381.003694:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767381.003704:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767381/real 1775767381]  req@ffff9f14852d0000 x1862022114692736/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767406 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767381.003732:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0000 x1862022114692736/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767406 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767381.003742:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767381.003746:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767381.003749:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114692736:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775767381.258765:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775767382.026695:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767382.282741:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775767383.050658:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767383.306796:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775767384.074678:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:0.0:1775767384.330681:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775767385.098724:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:0.0:1775767385.354744:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1775767386.122673:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:3.0:1775767386.122679:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:0.0:1775767386.122684:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1775767386.122689:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767386.122698:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4763
00000100:00080000:0.0:1775767386.122710:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767386.122718:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775767386.122767:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d2300 x1862022114692864/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767386.122793:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767386.122796:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775767386.122798:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767386.122801:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4763
00000100:00080000:0.0:1775767386.122805:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767386.122809:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767386.122817:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767386.122828:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114692864:192.168.202.110@tcp:250:kworker.0
00000100:00100000:0.0:1775767386.122831:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0000 x1862022114692992/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767386.122842:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767386.122845:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:0.0:1775767386.122846:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767386.122848:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4763
00000100:00080000:0.0:1775767386.122851:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767386.122853:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767386.122856:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767386.122864:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d1500 x1862022114693120/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767386.122872:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767386.122875:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775767386.122876:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767386.122879:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4763
00000100:00080000:0.0:1775767386.122888:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767386.122891:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767386.122895:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:3.0:1775767386.122898:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767386.122906:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3800 x1862022114693248/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767386.122914:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767386.122917:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1775767386.122918:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767386.122920:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4763
00000100:00080000:0.0:1775767386.122922:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767386.122925:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767386.122928:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775767386.122937:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d2a00 x1862022114693376/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767386.122944:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767386.122947:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775767386.122948:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767386.122952:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4763
00000100:00100000:3.0:1775767386.122952:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767386/real 1775767386]  req@ffff9f14a04d2300 x1862022114692864/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767411 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767386.122956:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767386.122962:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767386.122968:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767386.122980:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3100 x1862022114693504/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775767386.122988:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d2300 x1862022114692864/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767411 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767386.122989:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767386.122997:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:0.0:1775767386.122999:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767386.123007:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4763
00000100:00080000:0.0:1775767386.123012:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767386.123014:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767386.123015:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767386.123023:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00080000:3.0:1775767386.123025:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767386.123038:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114692864:192.168.202.110@tcp:250:kworker.0
00000100:00100000:0.0:1775767386.123058:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0a80 x1862022114693632/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767386.123070:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767386.123074:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775767386.123078:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:3.0:1775767386.123078:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114692992:192.168.202.110@tcp:38:kworker.0
00000100:00080000:0.0:1775767386.123087:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4763
00000100:00080000:0.0:1775767386.123093:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767386.123098:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767386.123104:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:3.0:1775767386.123112:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767386.123120:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0e00 x1862022114693760/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775767386.123128:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767386/real 1775767386]  req@ffff9f14a04d0000 x1862022114692992/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767411 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767386.123146:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0000 x1862022114692992/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767411 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767386.123156:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767386.123170:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767386.123174:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114692992:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767386.123181:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114693120:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775767386.123204:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767386.123214:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767386/real 1775767386]  req@ffff9f14a04d1500 x1862022114693120/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767411 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767386.123227:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1500 x1862022114693120/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767411 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767386.123236:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767386.123239:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767386.123243:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114693120:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767386.123258:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114693248:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775767386.123277:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767386.123287:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767386/real 1775767386]  req@ffff9f14a04d3800 x1862022114693248/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767411 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767386.123299:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3800 x1862022114693248/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767411 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767386.123307:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767386.123310:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767386.123313:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114693248:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775767386.123320:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114693376:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775767386.123339:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767386.123360:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767386/real 1775767386]  req@ffff9f14a04d2a00 x1862022114693376/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767411 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767386.123380:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d2a00 x1862022114693376/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767411 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767386.123388:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767386.123392:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767386.123395:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114693376:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775767386.123402:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114693504:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775767386.123422:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767386.123433:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767386/real 1775767386]  req@ffff9f14a04d3100 x1862022114693504/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767411 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767386.123445:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3100 x1862022114693504/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767411 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767386.123452:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767386.123455:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767386.123459:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114693504:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767386.123465:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114693632:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775767386.123484:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767386.123506:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767386/real 1775767386]  req@ffff9f14a04d0a80 x1862022114693632/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767411 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767386.123519:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0a80 x1862022114693632/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767411 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767386.123531:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767386.123534:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767386.123537:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114693632:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767386.123579:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114693760:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775767386.123602:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767386.123614:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767386/real 1775767386]  req@ffff9f14a04d0e00 x1862022114693760/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767411 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767386.123626:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0e00 x1862022114693760/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767411 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767386.123634:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767386.123638:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767386.123641:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114693760:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775767386.378694:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775767387.146655:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767387.402740:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775767388.170647:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767388.426662:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775767389.194634:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767389.450733:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775767390.218641:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767390.474786:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1775767391.242670:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767391.242678:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00100000:0.0:1775767391.242679:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:1.0:1775767391.242682:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767391.242691:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4768
00000100:00080000:1.0:1775767391.242701:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767391.242707:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775767391.242742:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff800 x1862022114693888/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767391.242771:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767391.242775:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775767391.242776:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767391.242780:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4768
00000100:00080000:1.0:1775767391.242783:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767391.242787:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767391.242796:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767391.242819:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fed80 x1862022114694016/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775767391.242825:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114693888:192.168.202.110@tcp:250:kworker.0
00000100:00080000:1.0:1775767391.242830:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767391.242834:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775767391.242836:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767391.242839:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4768
00000100:00080000:1.0:1775767391.242841:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767391.242843:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767391.242847:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767391.242856:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ffb80 x1862022114694144/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767391.242862:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767391.242864:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775767391.242866:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767391.242868:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4768
00000100:00080000:1.0:1775767391.242871:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767391.242873:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767391.242876:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775767391.242885:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fca80 x1862022114694272/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767391.242894:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000800:00000100:3.0:1775767391.242897:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:1.0:1775767391.242899:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1775767391.242903:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767391.242908:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4768
00000100:00080000:1.0:1775767391.242914:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767391.242919:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767391.242925:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775767391.242939:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fea00 x1862022114694400/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775767391.242942:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767391/real 1775767391]  req@ffff9f14ab4ff800 x1862022114693888/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767416 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767391.242955:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767391.242961:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775767391.242963:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767391.242970:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4768
00000100:00080000:1.0:1775767391.242975:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767391.242979:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775767391.242983:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff800 x1862022114693888/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767416 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:1.0:1775767391.242987:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00080000:3.0:1775767391.243029:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00100000:1.0:1775767391.243035:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fce00 x1862022114694528/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767391.243050:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767391.243060:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114693888:192.168.202.110@tcp:250:kworker.0
00000100:00080000:1.0:1775767391.243064:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767391.243068:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775767391.243069:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767391.243073:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4768
00000100:00080000:1.0:1775767391.243077:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767391.243080:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767391.243085:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767391.243089:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114694016:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775767391.243097:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdf80 x1862022114694656/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767391.243105:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767391.243107:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775767391.243109:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767391.243112:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4768
00000100:00080000:1.0:1775767391.243116:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000800:00000100:3.0:1775767391.243120:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:1.0:1775767391.243121:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767391.243127:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775767391.243136:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdc00 x1862022114694784/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775767391.243138:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767391/real 1775767391]  req@ffff9f14ab4fed80 x1862022114694016/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767416 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767391.243148:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fed80 x1862022114694016/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767416 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767391.243166:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767391.243170:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767391.243174:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114694016:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767391.243179:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ffb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114694144:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775767391.243192:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767391.243198:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767391/real 1775767391]  req@ffff9f14ab4ffb80 x1862022114694144/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767416 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767391.243205:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ffb80 x1862022114694144/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767416 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767391.243210:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767391.243212:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767391.243213:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ffb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114694144:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767391.243218:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114694272:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775767391.243228:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767391.243233:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767391/real 1775767391]  req@ffff9f14ab4fca80 x1862022114694272/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767416 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767391.243240:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fca80 x1862022114694272/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767416 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767391.243245:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767391.243247:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767391.243248:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114694272:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775767391.243252:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114694400:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775767391.243261:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767391.243266:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767391/real 1775767391]  req@ffff9f14ab4fea00 x1862022114694400/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767416 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767391.243273:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fea00 x1862022114694400/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767416 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767391.243277:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767391.243279:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767391.243281:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114694400:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775767391.243284:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114694528:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775767391.243293:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767391.243298:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767391/real 1775767391]  req@ffff9f14ab4fce00 x1862022114694528/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767416 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767391.243305:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fce00 x1862022114694528/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767416 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767391.243319:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767391.243320:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767391.243322:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114694528:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767391.243334:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114694656:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775767391.243343:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767391.243349:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767391/real 1775767391]  req@ffff9f14ab4fdf80 x1862022114694656/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767416 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767391.243356:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdf80 x1862022114694656/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767416 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767391.243361:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767391.243363:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767391.243364:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114694656:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767391.243368:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114694784:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775767391.243377:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767391.243382:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767391/real 1775767391]  req@ffff9f14ab4fdc00 x1862022114694784/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767416 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767391.243389:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdc00 x1862022114694784/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767416 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767391.243398:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767391.243400:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767391.243402:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114694784:192.168.202.110@tcp:8:kworker.0
00010000:00010000:0.0:1775767391.498674:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767392.266667:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:0.0:1775767392.522661:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767393.290636:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767393.546712:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767394.314660:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767394.570709:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767395.338660:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767395.594712:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767396.362660:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:1.0:1775767396.362682:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767396.362690:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1775767396.362696:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767396.362708:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4773
00000100:00080000:1.0:1775767396.362722:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767396.362731:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775767396.362781:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff100 x1862022114694912/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767396.362811:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767396.362814:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775767396.362816:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767396.362819:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4773
00000100:00080000:1.0:1775767396.362823:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767396.362826:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767396.362837:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767396.362851:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fe300 x1862022114695040/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767396.362860:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767396.362863:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775767396.362864:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767396.362869:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4773
00000100:00080000:1.0:1775767396.362875:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00100000:3.0:1775767396.362876:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114694912:192.168.202.110@tcp:250:kworker.0
00000100:00080000:1.0:1775767396.362881:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767396.362891:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767396.362899:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fc380 x1862022114695168/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767396.362905:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767396.362907:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775767396.362909:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767396.362912:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4773
00000100:00080000:1.0:1775767396.362914:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767396.362917:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767396.362920:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775767396.362930:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fd880 x1862022114695296/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767396.362938:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767396.362941:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1775767396.362942:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767396.362945:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4773
00000100:00080000:1.0:1775767396.362966:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767396.362970:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767396.362974:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775767396.362983:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdc00 x1862022114695424/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767396.362996:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767396.363001:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775767396.363003:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767396.363008:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4773
00000100:00080000:1.0:1775767396.363014:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767396.363018:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775767396.363024:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114695040:192.168.202.110@tcp:38:kworker.0
00000008:00100000:1.0:1775767396.363027:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767396.363046:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdf80 x1862022114695552/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767396.363058:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767396.363061:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775767396.363062:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767396.363067:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4773
00000100:00080000:1.0:1775767396.363071:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767396.363073:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767396.363079:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767396.363083:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114695168:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775767396.363091:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fce00 x1862022114695680/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767396.363098:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767396.363102:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775767396.363104:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767396.363109:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4773
00000100:00080000:1.0:1775767396.363112:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767396.363114:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767396.363118:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775767396.363122:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114695296:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775767396.363131:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fea00 x1862022114695808/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775767396.363153:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114695424:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775767396.363177:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114695552:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767396.363193:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114695680:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767396.363206:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114695808:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775767396.618724:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767397.386748:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767397.642791:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767398.410631:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:0.0:1775767398.666672:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767399.434841:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000400:00000100:1.0:1775767399.434855:0:2393:0:(lib-socket.c:454:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.202.110:988
00000400:00000100:1.0:1775767399.434897:0:2393:0:(acceptor.c:111:lnet_connect_console_error()) Connection to 192.168.202.110@tcp at host 192.168.202.110 was unreachable: the network or that node may be down, or Lustre may be misconfigured.
00000800:00000100:1.0:1775767399.434914:0:2393:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:1.0:1775767399.434964:0:2393:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:1.0:1775767399.434973:0:2393:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:1.0:1775767399.434981:0:2393:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:1.0:1775767399.434990:0:2393:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:1.0:1775767399.435000:0:2393:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:1.0:1775767399.435008:0:2393:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:1.0:1775767399.435016:0:2393:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000100:00100000:3.0:1775767399.435034:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767396/real 1775767399]  req@ffff9f14ab4fdf80 x1862022114695552/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767421 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767399.435062:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdf80 x1862022114695552/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767421 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767399.435089:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767399.435094:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767399.435099:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114695552:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767399.435109:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767396/real 1775767399]  req@ffff9f14ab4fce00 x1862022114695680/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767421 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767399.435126:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fce00 x1862022114695680/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767421 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767399.435134:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767399.435138:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767399.435141:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114695680:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767399.435147:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767396/real 1775767399]  req@ffff9f14ab4fea00 x1862022114695808/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767421 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767399.435160:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fea00 x1862022114695808/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767421 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767399.435169:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767399.435172:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767399.435175:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114695808:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775767399.435182:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767396/real 1775767399]  req@ffff9f14ab4fe300 x1862022114695040/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767421 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767399.435194:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fe300 x1862022114695040/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767421 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767399.435202:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767399.435206:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767399.435209:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114695040:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767399.435216:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767396/real 1775767399]  req@ffff9f14ab4fc380 x1862022114695168/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767421 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767399.435227:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fc380 x1862022114695168/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767421 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767399.435235:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767399.435238:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767399.435241:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114695168:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767399.435248:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767396/real 1775767399]  req@ffff9f14ab4fd880 x1862022114695296/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767421 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767399.435261:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fd880 x1862022114695296/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767421 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767399.435268:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767399.435272:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767399.435275:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114695296:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775767399.435282:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767396/real 1775767399]  req@ffff9f14ab4fdc00 x1862022114695424/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767421 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767399.435309:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdc00 x1862022114695424/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767421 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767399.435317:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767399.435322:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767399.435325:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114695424:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775767399.435334:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767396/real 1775767399]  req@ffff9f14ab4ff100 x1862022114694912/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767421 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767399.435345:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff100 x1862022114694912/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767421 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767399.435353:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767399.435356:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767399.435359:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114694912:192.168.202.110@tcp:250:kworker.0
00010000:00010000:0.0:1775767399.690659:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767400.458678:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767400.714828:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1775767401.482666:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767401.482676:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1775767401.482683:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767401.482694:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4778
00000100:00100000:0.0:1775767401.482702:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:1.0:1775767401.482706:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767401.482715:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775767401.482752:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fca80 x1862022114695936/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767401.482778:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767401.482782:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775767401.482788:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767401.482793:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4778
00000100:00080000:1.0:1775767401.482797:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767401.482802:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767401.482812:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767401.482826:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ffb80 x1862022114696064/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767401.482836:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:3.0:1775767401.482837:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114695936:192.168.202.110@tcp:250:kworker.0
00000100:00080000:1.0:1775767401.482841:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775767401.482843:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767401.482846:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4778
00000100:00080000:1.0:1775767401.482850:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767401.482853:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767401.482858:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767401.482866:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fed80 x1862022114696192/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767401.482880:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767401.482883:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775767401.482885:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767401.482888:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4778
00000100:00080000:1.0:1775767401.482891:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767401.482894:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000800:00000100:3.0:1775767401.482896:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000008:00100000:1.0:1775767401.482911:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775767401.482922:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff800 x1862022114696320/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767401.482931:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767401.482935:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1775767401.482937:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767401.482941:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4778
00000100:00100000:3.0:1775767401.482943:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767401/real 1775767401]  req@ffff9f14ab4fca80 x1862022114695936/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767426 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767401.482944:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767401.482948:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767401.482953:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775767401.482962:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff100 x1862022114696448/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775767401.482962:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fca80 x1862022114695936/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767426 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767401.482973:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767401.482974:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767401.482976:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775767401.482978:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767401.482978:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:1.0:1775767401.482982:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4778
00000100:00100000:3.0:1775767401.482983:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114695936:192.168.202.110@tcp:250:kworker.0
00000100:00080000:1.0:1775767401.482986:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767401.482990:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767401.482994:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767401.483003:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdc00 x1862022114696576/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775767401.483004:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ffb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114696064:192.168.202.110@tcp:38:kworker.0
00000100:00080000:1.0:1775767401.483013:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767401.483016:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775767401.483018:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767401.483022:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4778
00000100:00080000:1.0:1775767401.483025:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000800:00000100:3.0:1775767401.483026:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:1.0:1775767401.483029:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767401.483033:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767401.483037:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767401/real 1775767401]  req@ffff9f14ab4ffb80 x1862022114696064/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767426 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767401.483043:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fd880 x1862022114696704/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775767401.483051:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ffb80 x1862022114696064/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767426 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767401.483052:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767401.483056:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775767401.483058:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767401.483065:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4778
00000100:00080000:3.0:1775767401.483067:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767401.483069:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767401.483071:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00080000:1.0:1775767401.483073:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775767401.483075:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ffb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114696064:192.168.202.110@tcp:38:kworker.0
00000008:00100000:1.0:1775767401.483078:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775767401.483081:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114696192:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775767401.483088:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fc380 x1862022114696832/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:3.0:1775767401.483104:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767401.483114:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767401/real 1775767401]  req@ffff9f14ab4fed80 x1862022114696192/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767426 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767401.483127:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fed80 x1862022114696192/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767426 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767401.483135:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767401.483139:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767401.483142:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114696192:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767401.483149:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114696320:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775767401.483172:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767401.483182:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767401/real 1775767401]  req@ffff9f14ab4ff800 x1862022114696320/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767426 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767401.483195:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff800 x1862022114696320/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767426 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767401.483203:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767401.483206:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767401.483210:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114696320:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775767401.483216:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114696448:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775767401.483235:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767401.483246:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767401/real 1775767401]  req@ffff9f14ab4ff100 x1862022114696448/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767426 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767401.483257:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff100 x1862022114696448/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767426 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767401.483265:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767401.483275:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767401.483278:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114696448:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775767401.483295:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114696576:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775767401.483317:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767401.483328:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767401/real 1775767401]  req@ffff9f14ab4fdc00 x1862022114696576/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767426 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767401.483339:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdc00 x1862022114696576/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767426 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767401.483346:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767401.483349:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767401.483352:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114696576:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767401.483357:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114696704:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775767401.483374:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767401.483384:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767401/real 1775767401]  req@ffff9f14ab4fd880 x1862022114696704/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767426 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767401.483395:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fd880 x1862022114696704/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767426 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767401.483403:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767401.483406:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767401.483409:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114696704:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767401.483415:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114696832:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775767401.483433:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767401.483443:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767401/real 1775767401]  req@ffff9f14ab4fc380 x1862022114696832/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767426 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767401.483455:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fc380 x1862022114696832/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767426 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767401.483463:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767401.483959:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767401.483963:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114696832:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775767401.738765:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767402.506719:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767402.762768:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767403.530678:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767403.786764:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767404.554643:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767404.810781:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767405.578640:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:0.0:1775767405.834705:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767406.602666:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:2.0:1775767406.602690:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767406.602695:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1775767406.602699:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767406.602706:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4784
00000100:00080000:2.0:1775767406.602717:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767406.602723:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775767406.602756:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0380 x1862022114696960/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767406.602776:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767406.602779:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775767406.602780:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767406.602783:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4784
00000100:00080000:2.0:1775767406.602786:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767406.602789:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767406.602796:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775767406.602805:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1500 x1862022114697088/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767406.602819:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767406.602822:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775767406.602823:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767406.602829:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4784
00000100:00100000:3.0:1775767406.602830:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114696960:192.168.202.110@tcp:250:kworker.0
00000100:00080000:2.0:1775767406.602832:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767406.602838:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767406.602842:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775767406.602850:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2a00 x1862022114697216/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767406.602871:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767406.602873:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775767406.602874:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767406.602877:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4784
00000100:00080000:2.0:1775767406.602880:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767406.602883:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767406.602886:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775767406.602896:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1180 x1862022114697344/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767406.602903:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767406.602905:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1775767406.602906:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767406.602908:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4784
00000100:00080000:2.0:1775767406.602912:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767406.602915:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767406.602929:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775767406.602938:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2d80 x1862022114697472/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:3.0:1775767406.602940:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:2.0:1775767406.602948:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767406.602950:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775767406.602952:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767406.602955:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4784
00000100:00080000:2.0:1775767406.602957:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767406.602960:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767406.602963:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775767406.602970:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3800 x1862022114697600/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767406.602977:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767406.602981:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775767406.602982:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:3.0:1775767406.602982:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767406/real 1775767406]  req@ffff9f14852d0380 x1862022114696960/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767431 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767406.602988:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4784
00000100:00080000:2.0:1775767406.602992:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767406.602998:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767406.603003:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767406.603011:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0380 x1862022114696960/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767431 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767406.603016:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0a80 x1862022114697728/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767406.603029:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767406.603038:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767406.603047:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114696960:192.168.202.110@tcp:250:kworker.0
00000100:00080000:2.0:1775767406.603048:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767406.603056:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775767406.603058:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767406.603075:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4784
00000100:00080000:2.0:1775767406.603078:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767406.603081:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767406.603086:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775767406.603095:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0700 x1862022114697856/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775767406.603100:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114697088:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775767406.603131:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767406.603144:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767406/real 1775767406]  req@ffff9f14852d1500 x1862022114697088/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767431 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767406.603158:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1500 x1862022114697088/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767431 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767406.603167:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767406.603170:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767406.603174:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114697088:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767406.603180:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114697216:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775767406.603211:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767406.603222:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767406/real 1775767406]  req@ffff9f14852d2a00 x1862022114697216/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767431 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767406.603234:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2a00 x1862022114697216/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767431 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767406.603242:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767406.603245:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767406.603249:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114697216:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767406.603256:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114697344:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775767406.603277:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767406.603288:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767406/real 1775767406]  req@ffff9f14852d1180 x1862022114697344/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767431 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767406.603300:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1180 x1862022114697344/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767431 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767406.603308:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767406.603311:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767406.603315:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114697344:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775767406.603321:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114697472:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775767406.603353:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767406.603363:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767406/real 1775767406]  req@ffff9f14852d2d80 x1862022114697472/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767431 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767406.603375:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2d80 x1862022114697472/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767431 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767406.603383:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767406.603386:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767406.603389:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114697472:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775767406.603395:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114697600:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775767406.603414:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767406.603426:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767406/real 1775767406]  req@ffff9f14852d3800 x1862022114697600/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767431 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767406.603439:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3800 x1862022114697600/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767431 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767406.603447:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767406.603450:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767406.603453:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114697600:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767406.603468:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114697728:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775767406.603492:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767406.603515:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767406/real 1775767406]  req@ffff9f14852d0a80 x1862022114697728/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767431 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767406.603536:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0a80 x1862022114697728/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767431 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767406.603545:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767406.603548:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767406.603551:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114697728:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767406.603610:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114697856:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775767406.603632:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767406.603644:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767406/real 1775767406]  req@ffff9f14852d0700 x1862022114697856/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767431 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767406.603656:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0700 x1862022114697856/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767431 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767406.603681:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767406.603684:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767406.603688:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114697856:192.168.202.110@tcp:8:kworker.0
00010000:00010000:0.0:1775767406.858686:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767407.626632:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767407.882728:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767408.650650:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767408.906688:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767409.674621:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767409.930751:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767410.698746:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767410.954850:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1775767411.722802:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767411.722866:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1775767411.722872:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767411.722887:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4789
00000100:00080000:2.0:1775767411.722904:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00100000:3.0:1775767411.722913:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:2.0:1775767411.722914:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775767411.722965:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2680 x1862022114697984/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767411.722994:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767411.722998:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775767411.723001:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767411.723005:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4789
00000100:00080000:2.0:1775767411.723009:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767411.723015:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767411.723029:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775767411.723055:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1f80 x1862022114698112/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775767411.723055:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114697984:192.168.202.110@tcp:250:kworker.0
00000100:00080000:2.0:1775767411.723079:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767411.723083:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775767411.723084:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767411.723089:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4789
00000100:00080000:2.0:1775767411.723092:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767411.723094:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767411.723098:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775767411.723109:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0700 x1862022114698240/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767411.723116:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767411.723119:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775767411.723120:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767411.723123:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4789
00000100:00080000:2.0:1775767411.723126:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767411.723139:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767411.723144:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775767411.723157:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0a80 x1862022114698368/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767411.723168:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767411.723172:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1775767411.723175:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767411.723180:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4789
00000100:00080000:2.0:1775767411.723187:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767411.723192:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767411.723198:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:3.0:1775767411.723203:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767411.723215:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3800 x1862022114698496/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767411.723224:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767411.723227:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775767411.723228:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767411.723232:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4789
00000100:00080000:2.0:1775767411.723238:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767411.723242:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767411.723246:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767411.723252:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767411/real 1775767411]  req@ffff9f14852d2680 x1862022114697984/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767436 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767411.723259:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2d80 x1862022114698624/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767411.723280:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767411.723284:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775767411.723286:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767411.723292:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4789
00000100:00080000:2.0:1775767411.723297:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767411.723301:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767411.723307:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775767411.723320:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1180 x1862022114698752/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767411.723327:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767411.723329:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775767411.723331:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767411.723334:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4789
00000100:00080000:2.0:1775767411.723337:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767411.723339:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767411.723343:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775767411.723352:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2a00 x1862022114698880/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775767411.723391:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2680 x1862022114697984/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767436 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767411.723447:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767411.723457:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767411.723464:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114697984:192.168.202.110@tcp:250:kworker.0
00000100:00100000:3.0:1775767411.723499:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114698112:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775767411.723524:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767411.723534:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767411/real 1775767411]  req@ffff9f14852d1f80 x1862022114698112/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767436 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767411.723543:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1f80 x1862022114698112/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767436 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767411.723549:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767411.723552:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767411.723590:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114698112:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767411.723598:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114698240:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775767411.723627:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767411.723640:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767411/real 1775767411]  req@ffff9f14852d0700 x1862022114698240/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767436 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767411.723656:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0700 x1862022114698240/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767436 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767411.723665:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767411.723668:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767411.723671:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114698240:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767411.723677:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114698368:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775767411.723714:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767411.723727:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767411/real 1775767411]  req@ffff9f14852d0a80 x1862022114698368/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767436 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767411.723742:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0a80 x1862022114698368/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767436 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767411.723752:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767411.723756:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767411.723760:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114698368:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775767411.723784:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114698496:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775767411.723822:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767411.723849:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767411/real 1775767411]  req@ffff9f14852d3800 x1862022114698496/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767436 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767411.723857:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3800 x1862022114698496/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767436 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767411.723862:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767411.723864:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767411.723866:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114698496:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775767411.723870:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114698624:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775767411.723884:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767411.723890:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767411/real 1775767411]  req@ffff9f14852d2d80 x1862022114698624/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767436 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767411.723897:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2d80 x1862022114698624/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767436 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767411.723902:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767411.723904:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767411.723906:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114698624:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767411.723910:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114698752:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775767411.723919:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767411.723925:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767411/real 1775767411]  req@ffff9f14852d1180 x1862022114698752/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767436 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767411.723932:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1180 x1862022114698752/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767436 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767411.723937:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767411.723939:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767411.723941:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114698752:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767411.723944:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114698880:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775767411.723954:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767411.723959:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767411/real 1775767411]  req@ffff9f14852d2a00 x1862022114698880/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767436 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767411.723966:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2a00 x1862022114698880/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767436 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767411.723970:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767411.723972:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767411.723975:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114698880:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775767411.978710:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775767412.746651:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:0.0:1775767413.002704:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775767413.770675:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:0.0:1775767414.026678:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767414.794744:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767415.050745:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767415.818734:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767416.074734:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1775767416.842661:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:2.0:1775767416.842662:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:1.0:1775767416.842672:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1775767416.842678:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767416.842688:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4794
00000100:00080000:1.0:1775767416.842701:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767416.842707:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775767416.842769:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fe300 x1862022114699008/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767416.842795:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767416.842798:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775767416.842799:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767416.842803:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4794
00000100:00080000:1.0:1775767416.842806:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767416.842809:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767416.842819:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767416.842831:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fea00 x1862022114699136/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767416.842839:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767416.842842:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775767416.842845:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:3.0:1775767416.842848:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114699008:192.168.202.110@tcp:250:kworker.0
00000100:00080000:1.0:1775767416.842850:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4794
00000100:00080000:1.0:1775767416.842855:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767416.842860:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767416.842864:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767416.842872:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fce00 x1862022114699264/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767416.842879:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767416.842881:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775767416.842882:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767416.842886:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4794
00000100:00080000:1.0:1775767416.842888:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767416.842890:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767416.842893:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775767416.842900:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdf80 x1862022114699392/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767416.842907:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767416.842909:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1775767416.842909:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767416.842912:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4794
00000100:00080000:1.0:1775767416.842915:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767416.842917:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767416.842921:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775767416.842927:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fc380 x1862022114699520/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767416.842932:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767416.842934:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775767416.842937:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767416.842943:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4794
00000800:00000100:3.0:1775767416.842943:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:1.0:1775767416.842949:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767416.842952:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767416.842955:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767416.842963:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fd880 x1862022114699648/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767416.842968:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767416.842971:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775767416.842973:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767416.842976:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4794
00000100:00080000:1.0:1775767416.842978:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767416.842980:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767416.842982:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767416.842990:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdc00 x1862022114699776/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767416.842995:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767416.842998:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775767416.842999:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767416.843004:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4794
00000100:00100000:3.0:1775767416.843006:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767416/real 1775767416]  req@ffff9f14ab4fe300 x1862022114699008/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767441 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767416.843007:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767416.843020:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767416.843023:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775767416.843036:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff100 x1862022114699904/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775767416.843036:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fe300 x1862022114699008/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767441 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767416.843062:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767416.843070:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767416.843087:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114699008:192.168.202.110@tcp:250:kworker.0
00000100:00100000:3.0:1775767416.843115:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114699136:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775767416.843142:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767416.843156:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767416/real 1775767416]  req@ffff9f14ab4fea00 x1862022114699136/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767441 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767416.843169:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fea00 x1862022114699136/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767441 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767416.843179:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767416.843183:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767416.843187:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114699136:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767416.843193:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114699264:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775767416.843215:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767416.843227:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767416/real 1775767416]  req@ffff9f14ab4fce00 x1862022114699264/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767441 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767416.843240:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fce00 x1862022114699264/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767441 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767416.843248:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767416.843251:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767416.843255:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114699264:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767416.843262:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114699392:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775767416.843284:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767416.843294:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767416/real 1775767416]  req@ffff9f14ab4fdf80 x1862022114699392/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767441 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767416.843307:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdf80 x1862022114699392/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767441 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767416.843316:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767416.843320:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767416.843323:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114699392:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775767416.843329:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114699520:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775767416.843345:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767416.843355:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767416/real 1775767416]  req@ffff9f14ab4fc380 x1862022114699520/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767441 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767416.843367:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fc380 x1862022114699520/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767441 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767416.843375:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767416.843379:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767416.843382:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114699520:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775767416.843388:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114699648:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775767416.843406:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767416.843416:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767416/real 1775767416]  req@ffff9f14ab4fd880 x1862022114699648/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767441 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767416.843436:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fd880 x1862022114699648/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767441 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767416.843444:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767416.843448:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767416.843451:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114699648:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767416.843457:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114699776:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775767416.843477:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767416.843488:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767416/real 1775767416]  req@ffff9f14ab4fdc00 x1862022114699776/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767441 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767416.843516:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdc00 x1862022114699776/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767441 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767416.843525:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767416.843528:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767416.843532:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114699776:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767416.843538:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114699904:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775767416.843557:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767416.843567:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767416/real 1775767416]  req@ffff9f14ab4ff100 x1862022114699904/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767441 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767416.843604:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff100 x1862022114699904/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767441 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767416.843618:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767416.843621:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767416.843625:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114699904:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775767417.098795:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767417.866726:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767418.122792:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775767418.890772:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767419.146728:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775767419.914672:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:0.0:1775767420.170793:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775767420.938649:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:0.0:1775767421.194654:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1775767421.962678:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767421.962688:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1775767421.962692:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767421.962703:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4799
00000100:00080000:1.0:1775767421.962725:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767421.962733:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775767421.962765:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff800 x1862022114700032/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767421.962787:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767421.962790:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775767421.962791:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767421.962795:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4799
00000100:00080000:1.0:1775767421.962797:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767421.962800:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767421.962807:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767421.962816:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fed80 x1862022114700160/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767421.962822:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767421.962825:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775767421.962826:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767421.962828:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4799
00000100:00080000:1.0:1775767421.962831:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767421.962835:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767421.962848:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767421.962863:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ffb80 x1862022114700288/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767421.962873:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767421.962877:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775767421.962879:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767421.962882:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4799
00000100:00080000:1.0:1775767421.962886:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767421.962890:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767421.962894:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775767421.962902:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fca80 x1862022114700416/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767421.962908:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767421.962910:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1775767421.962912:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767421.962915:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4799
00000100:00080000:1.0:1775767421.962919:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767421.962923:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767421.962927:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775767421.962936:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff100 x1862022114700544/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767421.962946:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767421.962950:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775767421.962952:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767421.962956:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4799
00000100:00080000:1.0:1775767421.962959:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767421.962963:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767421.962968:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767421.962978:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdc00 x1862022114700672/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767421.962984:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767421.962986:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775767421.962988:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767421.962990:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4799
00000100:00080000:1.0:1775767421.962992:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767421.962995:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767421.962998:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767421.963003:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fd880 x1862022114700800/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767421.963008:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767421.963010:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775767421.963011:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767421.963013:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4799
00000100:00080000:1.0:1775767421.963015:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767421.963017:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767421.963020:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775767421.963026:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fc380 x1862022114700928/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775767421.963138:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114700032:192.168.202.110@tcp:250:kworker.0
00000100:00100000:2.0:1775767421.963173:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000800:00000100:3.0:1775767421.963210:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767421.963244:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767421/real 1775767421]  req@ffff9f14ab4ff800 x1862022114700032/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767446 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767421.963262:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff800 x1862022114700032/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767446 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767421.963272:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767421.963276:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767421.963280:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114700032:192.168.202.110@tcp:250:kworker.0
00000100:00100000:3.0:1775767421.963288:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114700160:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775767421.963307:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767421.963316:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767421/real 1775767421]  req@ffff9f14ab4fed80 x1862022114700160/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767446 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767421.963328:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fed80 x1862022114700160/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767446 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767421.963343:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767421.963346:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767421.963349:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114700160:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767421.963356:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ffb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114700288:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775767421.963376:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767421.963386:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767421/real 1775767421]  req@ffff9f14ab4ffb80 x1862022114700288/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767446 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767421.963399:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ffb80 x1862022114700288/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767446 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767421.963407:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767421.963410:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767421.963413:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ffb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114700288:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767421.963420:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114700416:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775767421.963441:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767421.963451:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767421/real 1775767421]  req@ffff9f14ab4fca80 x1862022114700416/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767446 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767421.963460:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fca80 x1862022114700416/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767446 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767421.963465:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767421.963467:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767421.963469:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114700416:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775767421.963473:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114700544:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775767421.963486:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767421.963492:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767421/real 1775767421]  req@ffff9f14ab4ff100 x1862022114700544/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767446 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767421.963499:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff100 x1862022114700544/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767446 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767421.963504:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767421.963506:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767421.963507:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114700544:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775767421.963511:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114700672:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775767421.963520:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767421.963525:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767421/real 1775767421]  req@ffff9f14ab4fdc00 x1862022114700672/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767446 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767421.963533:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdc00 x1862022114700672/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767446 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767421.963537:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767421.963539:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767421.963541:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114700672:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767421.963544:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114700800:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775767421.963571:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767421.963579:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767421/real 1775767421]  req@ffff9f14ab4fd880 x1862022114700800/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767446 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767421.963587:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fd880 x1862022114700800/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767446 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767421.963592:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767421.963594:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767421.963596:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114700800:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767421.963600:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114700928:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775767421.963617:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767421.963626:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767421/real 1775767421]  req@ffff9f14ab4fc380 x1862022114700928/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767446 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767421.963639:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fc380 x1862022114700928/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767446 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767421.963647:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767421.963651:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767421.963654:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114700928:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775767422.218727:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767422.986660:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767423.242808:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767424.010710:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767424.266737:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767425.034649:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767425.290764:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775767426.058718:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767426.314746:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775767427.082652:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:0.0:1775767427.082688:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767427.082694:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1775767427.082704:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767427.082712:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4804
00000100:00080000:0.0:1775767427.082721:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767427.082726:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775767427.082758:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d1f80 x1862022114701056/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767427.082783:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767427.082786:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775767427.082788:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767427.082791:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4804
00000100:00080000:0.0:1775767427.082793:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767427.082796:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767427.082803:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767427.082811:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0380 x1862022114701184/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767427.082817:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767427.082820:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:0.0:1775767427.082821:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767427.082823:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4804
00000100:00080000:0.0:1775767427.082825:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767427.082833:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767427.082836:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767427.082842:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3480 x1862022114701312/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767427.082849:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767427.082852:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775767427.082853:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767427.082856:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4804
00000100:00080000:0.0:1775767427.082859:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00100000:3.0:1775767427.082861:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114701056:192.168.202.110@tcp:250:kworker.0
00000100:00080000:0.0:1775767427.082864:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767427.082867:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775767427.082874:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d1180 x1862022114701440/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767427.082881:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767427.082883:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1775767427.082885:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767427.082888:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4804
00000100:00080000:0.0:1775767427.082890:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767427.082892:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767427.082896:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775767427.082904:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3b80 x1862022114701568/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767427.082910:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767427.082913:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775767427.082914:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767427.082917:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4804
00000100:00080000:0.0:1775767427.082919:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767427.082922:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767427.082925:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000800:00000100:3.0:1775767427.082929:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767427.082937:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0e00 x1862022114701696/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767427.082948:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767427.082951:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:0.0:1775767427.082952:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767427.082955:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4804
00000100:00080000:0.0:1775767427.082957:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767427.082960:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767427.082964:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767427.082965:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767427/real 1775767427]  req@ffff9f14a04d1f80 x1862022114701056/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767452 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767427.082975:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0a80 x1862022114701824/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767427.082997:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767427.083000:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775767427.083002:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767427.083006:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4804
00000100:00100000:3.0:1775767427.083006:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1f80 x1862022114701056/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767452 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767427.083011:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767427.083018:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767427.083024:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00080000:3.0:1775767427.083025:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767427.083031:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767427.083034:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3100 x1862022114701952/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775767427.083038:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114701056:192.168.202.110@tcp:250:kworker.0
00000100:00100000:3.0:1775767427.083051:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114701184:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775767427.083070:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767427.083078:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767427/real 1775767427]  req@ffff9f14a04d0380 x1862022114701184/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767452 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767427.083087:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0380 x1862022114701184/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767452 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767427.083093:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767427.083095:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767427.083097:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114701184:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767427.083101:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114701312:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775767427.083120:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767427.083127:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767427/real 1775767427]  req@ffff9f14a04d3480 x1862022114701312/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767452 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767427.083135:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3480 x1862022114701312/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767452 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767427.083145:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767427.083147:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767427.083150:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114701312:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767427.083169:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114701440:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775767427.083188:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767427.083194:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767427/real 1775767427]  req@ffff9f14a04d1180 x1862022114701440/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767452 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767427.083202:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1180 x1862022114701440/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767452 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767427.083208:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767427.083210:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767427.083212:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114701440:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775767427.083216:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114701568:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775767427.083226:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767427.083231:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767427/real 1775767427]  req@ffff9f14a04d3b80 x1862022114701568/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767452 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767427.083239:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3b80 x1862022114701568/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767452 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767427.083244:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767427.083245:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767427.083248:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114701568:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775767427.083252:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114701696:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775767427.083261:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767427.083266:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767427/real 1775767427]  req@ffff9f14a04d0e00 x1862022114701696/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767452 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767427.083275:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0e00 x1862022114701696/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767452 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767427.083283:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767427.083286:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767427.083290:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114701696:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767427.083295:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114701824:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775767427.083324:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767427.083334:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767427/real 1775767427]  req@ffff9f14a04d0a80 x1862022114701824/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767452 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767427.083343:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0a80 x1862022114701824/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767452 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767427.083348:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767427.083350:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767427.083352:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114701824:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767427.083356:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114701952:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775767427.083368:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767427.083373:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767427/real 1775767427]  req@ffff9f14a04d3100 x1862022114701952/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767452 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767427.083381:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3100 x1862022114701952/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767452 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767427.083386:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767427.083387:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767427.083389:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114701952:192.168.202.110@tcp:8:kworker.0
00010000:00010000:0.0:1775767427.338689:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775767428.106694:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:0.0:1775767428.362714:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775767429.130624:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767429.386726:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775767430.154629:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767430.410699:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775767431.178682:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767431.434690:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775767432.202663:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:1.0:1775767432.202675:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767432.202681:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1775767432.202684:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767432.202693:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4809
00000100:00080000:1.0:1775767432.202703:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767432.202710:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775767432.202745:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdf80 x1862022114702080/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767432.202771:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767432.202774:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775767432.202776:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767432.202780:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4809
00000100:00080000:1.0:1775767432.202784:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767432.202789:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775767432.202798:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114702080:192.168.202.110@tcp:250:kworker.0
00000008:00100000:1.0:1775767432.202799:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767432.202817:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fce00 x1862022114702208/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767432.202828:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767432.202831:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775767432.202832:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767432.202836:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4809
00000100:00080000:1.0:1775767432.202841:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767432.202845:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000800:00000100:3.0:1775767432.202849:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000008:00100000:1.0:1775767432.202851:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767432.202865:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fea00 x1862022114702336/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767432.202875:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767432.202878:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775767432.202881:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:3.0:1775767432.202881:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767432/real 1775767432]  req@ffff9f14ab4fdf80 x1862022114702080/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767457 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767432.202890:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4809
00000100:00080000:1.0:1775767432.202897:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00100000:3.0:1775767432.202901:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdf80 x1862022114702080/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767457 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767432.202902:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767432.202912:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00080000:3.0:1775767432.202913:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767432.202919:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767432.202922:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114702080:192.168.202.110@tcp:250:kworker.0
00000100:00100000:1.0:1775767432.202929:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fe300 x1862022114702464/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767432.202940:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:3.0:1775767432.202940:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114702208:192.168.202.110@tcp:38:kworker.0
00000100:00080000:1.0:1775767432.202948:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1775767432.202951:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767432.202956:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4809
00000800:00000100:3.0:1775767432.202960:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:1.0:1775767432.202961:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767432.202967:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767432.202973:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775767432.202977:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767432/real 1775767432]  req@ffff9f14ab4fce00 x1862022114702208/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767457 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767432.202988:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdf80 x1862022114702592/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775767432.203015:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fce00 x1862022114702208/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767457 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767432.203018:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767432.203026:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775767432.203030:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767432.203036:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767432.203040:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767432.203044:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114702208:192.168.202.110@tcp:38:kworker.0
00000100:00080000:1.0:1775767432.203045:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4809
00000100:00080000:1.0:1775767432.203052:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:3.0:1775767432.203053:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114702336:192.168.202.110@tcp:38:kworker.0
00000100:00080000:1.0:1775767432.203059:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767432.203065:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000800:00000100:3.0:1775767432.203079:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767432.203080:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fc380 x1862022114702720/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775767432.203090:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767432/real 1775767432]  req@ffff9f14ab4fea00 x1862022114702336/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767457 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767432.203092:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767432.203101:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775767432.203104:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:3.0:1775767432.203107:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fea00 x1862022114702336/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767457 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767432.203113:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767432.203116:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00080000:1.0:1775767432.203118:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4809
00000100:00100000:3.0:1775767432.203119:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114702336:192.168.202.110@tcp:38:kworker.0
00000100:00080000:1.0:1775767432.203130:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767432.203136:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775767432.203138:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114702464:192.168.202.110@tcp:8:kworker.0
00000008:00100000:1.0:1775767432.203145:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000800:00000100:3.0:1775767432.203154:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767432.203161:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fd880 x1862022114702848/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775767432.203164:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767432/real 1775767432]  req@ffff9f14ab4fe300 x1862022114702464/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767457 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767432.203189:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fe300 x1862022114702464/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767457 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767432.203192:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767432.203201:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775767432.203202:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767432.203204:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767432.203206:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767432.203211:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114702464:192.168.202.110@tcp:8:kworker.0
00000100:00080000:1.0:1775767432.203215:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4809
00000100:00100000:3.0:1775767432.203218:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114702592:192.168.202.110@tcp:8:kworker.0
00000100:00080000:1.0:1775767432.203221:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767432.203227:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000800:00000100:3.0:1775767432.203231:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000008:00100000:1.0:1775767432.203234:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775767432.203239:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767432/real 1775767432]  req@ffff9f14ab4fdf80 x1862022114702592/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767457 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767432.203248:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdf80 x1862022114702592/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767457 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767432.203251:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdc00 x1862022114702976/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767432.203260:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767432.203269:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767432.203275:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114702592:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775767432.203290:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114702720:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775767432.203309:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767432.203319:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767432/real 1775767432]  req@ffff9f14ab4fc380 x1862022114702720/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767457 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767432.203326:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fc380 x1862022114702720/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767457 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767432.203331:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767432.203332:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767432.203335:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114702720:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767432.203344:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114702848:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775767432.203355:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767432.203359:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767432/real 1775767432]  req@ffff9f14ab4fd880 x1862022114702848/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767457 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767432.203366:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fd880 x1862022114702848/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767457 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767432.203370:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767432.203372:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767432.203374:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114702848:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767432.203376:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114702976:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775767432.203384:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767432.203388:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767432/real 1775767432]  req@ffff9f14ab4fdc00 x1862022114702976/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767457 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767432.203394:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdc00 x1862022114702976/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767457 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767432.203398:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767432.203400:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767432.203401:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114702976:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775767432.458717:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775767433.226679:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767433.482702:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775767434.250640:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:0.0:1775767434.506674:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767435.274774:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:0.0:1775767435.530784:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767436.298667:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767436.554725:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767437.322695:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:2.0:1775767437.322728:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767437.322734:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1775767437.322738:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767437.322747:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4814
00000100:00080000:2.0:1775767437.322759:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767437.322765:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775767437.322801:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1500 x1862022114703104/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767437.322829:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767437.322832:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775767437.322834:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767437.322838:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4814
00000100:00080000:2.0:1775767437.322842:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767437.322846:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767437.322855:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775767437.322868:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0380 x1862022114703232/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775767437.322871:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114703104:192.168.202.110@tcp:250:kworker.0
00000100:00080000:2.0:1775767437.322886:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767437.322889:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775767437.322891:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767437.322896:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4814
00000100:00080000:2.0:1775767437.322899:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767437.322903:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767437.322908:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775767437.322919:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0000 x1862022114703360/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767437.322927:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767437.322932:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000800:00000100:3.0:1775767437.322933:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:2.0:1775767437.322935:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767437.322940:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4814
00000100:00080000:2.0:1775767437.322945:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767437.322949:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767437.322955:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775767437.322965:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3b80 x1862022114703488/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775767437.322972:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767437/real 1775767437]  req@ffff9f14852d1500 x1862022114703104/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767462 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767437.323007:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767437.323011:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00100000:3.0:1775767437.323013:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1500 x1862022114703104/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767462 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767437.323015:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767437.323023:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4814
00000100:00080000:2.0:1775767437.323029:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767437.323029:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767437.323037:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00080000:3.0:1775767437.323038:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000008:00100000:2.0:1775767437.323043:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775767437.323043:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114703104:192.168.202.110@tcp:250:kworker.0
00000100:00100000:2.0:1775767437.323060:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0e00 x1862022114703616/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775767437.323068:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114703232:192.168.202.110@tcp:38:kworker.0
00000100:00080000:2.0:1775767437.323069:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767437.323074:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775767437.323075:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767437.323079:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4814
00000100:00080000:2.0:1775767437.323086:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000800:00000100:3.0:1775767437.323091:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:2.0:1775767437.323092:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767437.323098:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767437.323106:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767437/real 1775767437]  req@ffff9f14852d0380 x1862022114703232/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767462 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767437.323109:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1880 x1862022114703744/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767437.323130:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:3.0:1775767437.323132:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0380 x1862022114703232/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767462 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767437.323133:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775767437.323136:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767437.323143:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4814
00000100:00080000:2.0:1775767437.323148:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767437.323149:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767437.323152:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00080000:3.0:1775767437.323156:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000008:00100000:2.0:1775767437.323157:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767437.323162:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114703232:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767437.323167:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3100 x1862022114703872/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775767437.323170:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114703360:192.168.202.110@tcp:38:kworker.0
00000100:00080000:2.0:1775767437.323182:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767437.323185:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775767437.323187:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767437.323190:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4814
00000800:00000100:3.0:1775767437.323194:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:2.0:1775767437.323195:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767437.323200:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767437.323204:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775767437.323206:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767437/real 1775767437]  req@ffff9f14852d0000 x1862022114703360/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767462 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767437.323214:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2300 x1862022114704000/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775767437.323236:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0000 x1862022114703360/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767462 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767437.323255:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767437.323260:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767437.323264:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114703360:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767437.323270:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114703488:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775767437.323293:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767437.323305:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767437/real 1775767437]  req@ffff9f14852d3b80 x1862022114703488/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767462 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767437.323316:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3b80 x1862022114703488/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767462 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767437.323322:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767437.323324:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767437.323326:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114703488:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775767437.323338:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114703616:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775767437.323360:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767437.323367:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767437/real 1775767437]  req@ffff9f14852d0e00 x1862022114703616/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767462 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767437.323375:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0e00 x1862022114703616/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767462 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767437.323384:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767437.323386:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767437.323388:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114703616:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775767437.323392:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114703744:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775767437.323402:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767437.323412:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767437/real 1775767437]  req@ffff9f14852d1880 x1862022114703744/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767462 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767437.323420:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1880 x1862022114703744/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767462 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767437.323425:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767437.323427:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767437.323429:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114703744:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767437.323432:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114703872:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775767437.323442:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767437.323447:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767437/real 1775767437]  req@ffff9f14852d3100 x1862022114703872/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767462 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767437.323454:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3100 x1862022114703872/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767462 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767437.323459:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767437.323461:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767437.323464:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114703872:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767437.323467:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114704000:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775767437.323477:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767437.323483:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767437/real 1775767437]  req@ffff9f14852d2300 x1862022114704000/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767462 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767437.323495:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2300 x1862022114704000/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767462 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767437.323503:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767437.323506:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767437.323509:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114704000:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775767437.578717:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767438.346664:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767438.602732:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767439.370689:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767439.626751:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767440.394693:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767440.650709:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767441.418653:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:0.0:1775767441.674682:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1775767442.442645:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:3.0:1775767442.442650:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:2.0:1775767442.442657:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1775767442.442670:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767442.442680:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4819
00000100:00080000:2.0:1775767442.442692:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767442.442699:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775767442.442734:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3480 x1862022114704128/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767442.442756:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767442.442759:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775767442.442760:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767442.442763:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4819
00000100:00080000:2.0:1775767442.442766:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767442.442770:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767442.442778:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775767442.442787:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1c00 x1862022114704256/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775767442.442792:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114704128:192.168.202.110@tcp:250:kworker.0
00000100:00080000:2.0:1775767442.442803:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767442.442806:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775767442.442807:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767442.442810:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4819
00000100:00080000:2.0:1775767442.442811:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767442.442814:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767442.442817:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775767442.442826:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2300 x1862022114704384/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767442.442833:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767442.442835:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775767442.442837:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767442.442839:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4819
00000100:00080000:2.0:1775767442.442842:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767442.442844:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767442.442847:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775767442.442856:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3100 x1862022114704512/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:3.0:1775767442.442856:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:2.0:1775767442.442865:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767442.442867:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1775767442.442869:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767442.442872:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4819
00000100:00080000:2.0:1775767442.442875:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767442.442877:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767442.442880:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775767442.442886:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1880 x1862022114704640/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767442.442892:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767442.442895:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775767442.442897:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767442.442900:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4819
00000100:00100000:3.0:1775767442.442904:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767442/real 1775767442]  req@ffff9f14852d3480 x1862022114704128/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767467 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767442.442905:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767442.442913:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767442.442921:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775767442.442932:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0e00 x1862022114704768/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775767442.442932:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3480 x1862022114704128/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767467 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767442.442961:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767442.442965:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775767442.442966:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767442.442968:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767442.442970:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4819
00000100:00080000:2.0:1775767442.442975:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767442.442979:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00080000:3.0:1775767442.442979:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000008:00100000:2.0:1775767442.442986:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767442.442990:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114704128:192.168.202.110@tcp:250:kworker.0
00000100:00100000:2.0:1775767442.442995:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3b80 x1862022114704896/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767442.443004:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767442.443006:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775767442.443007:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767442.443010:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4819
00000100:00080000:2.0:1775767442.443012:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767442.443015:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767442.443018:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775767442.443019:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114704256:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767442.443026:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0000 x1862022114705024/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:3.0:1775767442.443049:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767442.443061:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767442/real 1775767442]  req@ffff9f14852d1c00 x1862022114704256/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767467 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767442.443071:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1c00 x1862022114704256/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767467 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767442.443076:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767442.443079:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767442.443081:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114704256:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767442.443085:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114704384:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775767442.443096:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767442.443102:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767442/real 1775767442]  req@ffff9f14852d2300 x1862022114704384/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767467 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767442.443109:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2300 x1862022114704384/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767467 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767442.443114:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767442.443116:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767442.443117:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114704384:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767442.443121:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114704512:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775767442.443131:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767442.443136:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767442/real 1775767442]  req@ffff9f14852d3100 x1862022114704512/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767467 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767442.443149:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3100 x1862022114704512/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767467 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767442.443154:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767442.443155:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767442.443157:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114704512:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775767442.443161:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114704640:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775767442.443170:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767442.443175:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767442/real 1775767442]  req@ffff9f14852d1880 x1862022114704640/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767467 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767442.443182:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1880 x1862022114704640/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767467 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767442.443187:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767442.443197:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767442.443199:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114704640:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775767442.443202:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114704768:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775767442.443213:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767442.443220:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767442/real 1775767442]  req@ffff9f14852d0e00 x1862022114704768/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767467 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767442.443227:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0e00 x1862022114704768/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767467 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767442.443236:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767442.443238:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767442.443241:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114704768:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767442.443244:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114704896:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775767442.443252:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767442.443257:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767442/real 1775767442]  req@ffff9f14852d3b80 x1862022114704896/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767467 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767442.443264:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3b80 x1862022114704896/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767467 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767442.443269:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767442.443271:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767442.443273:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114704896:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767442.443286:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114705024:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775767442.443295:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767442.443301:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767442/real 1775767442]  req@ffff9f14852d0000 x1862022114705024/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767467 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767442.443309:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0000 x1862022114705024/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767467 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767442.443313:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767442.443315:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767442.443317:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114705024:192.168.202.110@tcp:8:kworker.0
00010000:00010000:0.0:1775767442.698650:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775767443.466610:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767443.722702:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775767444.490667:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767444.746744:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767445.514701:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767445.770731:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767446.538639:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767446.794781:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775767447.562738:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:2.0:1775767447.562739:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767447.562751:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1775767447.562757:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767447.562771:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4825
00000100:00080000:2.0:1775767447.562789:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767447.562800:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775767447.562849:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0380 x1862022114705152/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767447.562886:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767447.562891:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775767447.562893:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767447.562900:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4825
00000100:00080000:2.0:1775767447.562905:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767447.562911:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767447.562923:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775767447.562945:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1500 x1862022114705280/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767447.562957:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767447.562960:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775767447.562962:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767447.562966:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4825
00000100:00080000:2.0:1775767447.562974:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767447.562979:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775767447.562985:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114705152:192.168.202.110@tcp:250:kworker.0
00000008:00100000:2.0:1775767447.562988:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775767447.563004:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2a00 x1862022114705408/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767447.563012:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767447.563014:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775767447.563016:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767447.563018:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4825
00000100:00080000:2.0:1775767447.563021:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767447.563023:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767447.563026:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775767447.563036:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1180 x1862022114705536/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767447.563042:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767447.563046:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1775767447.563048:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767447.563051:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4825
00000100:00080000:2.0:1775767447.563054:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767447.563060:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000800:00000100:0.0:1775767447.563062:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000008:00100000:2.0:1775767447.563066:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775767447.563079:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2d80 x1862022114705664/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767447.563091:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767447.563095:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775767447.563098:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767447.563101:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4825
00000100:00080000:2.0:1775767447.563104:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767447.563107:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767447.563110:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767447.563113:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767447/real 1775767447]  req@ffff9f14852d0380 x1862022114705152/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767472 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767447.563121:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3800 x1862022114705792/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767447.563130:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767447.563132:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775767447.563134:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775767447.563137:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0380 x1862022114705152/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767472 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767447.563140:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4825
00000100:00080000:2.0:1775767447.563146:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767447.563152:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767447.563155:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00080000:0.0:1775767447.563156:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767447.563162:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767447.563163:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0a80 x1862022114705920/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775767447.563168:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114705152:192.168.202.110@tcp:250:kworker.0
00000100:00080000:2.0:1775767447.563178:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1775767447.563181:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114705280:192.168.202.110@tcp:38:kworker.0
00000100:00080000:2.0:1775767447.563182:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775767447.563186:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767447.563191:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4825
00000100:00080000:2.0:1775767447.563194:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767447.563197:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767447.563201:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:0.0:1775767447.563207:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767447.563211:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0700 x1862022114706048/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775767447.563220:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767447/real 1775767447]  req@ffff9f14852d1500 x1862022114705280/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767472 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767447.563230:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1500 x1862022114705280/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767472 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767447.563237:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767447.563240:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767447.563243:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114705280:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767447.563264:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114705408:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767447.563280:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767447.563287:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767447/real 1775767447]  req@ffff9f14852d2a00 x1862022114705408/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767472 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767447.563295:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2a00 x1862022114705408/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767472 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767447.563301:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767447.563303:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767447.563305:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114705408:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767447.563313:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114705536:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767447.563328:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767447.563336:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767447/real 1775767447]  req@ffff9f14852d1180 x1862022114705536/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767472 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767447.563344:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1180 x1862022114705536/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767472 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767447.563350:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767447.563352:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767447.563356:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114705536:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775767447.563361:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114705664:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767447.563374:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767447.563380:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767447/real 1775767447]  req@ffff9f14852d2d80 x1862022114705664/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767472 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767447.563387:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2d80 x1862022114705664/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767472 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767447.563397:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767447.563400:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767447.563402:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114705664:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775767447.563407:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114705792:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767447.563419:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767447.563427:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767447/real 1775767447]  req@ffff9f14852d3800 x1862022114705792/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767472 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767447.563435:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3800 x1862022114705792/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767472 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767447.563441:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767447.563443:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767447.563445:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114705792:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767447.563460:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114705920:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767447.563500:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767447.563512:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767447/real 1775767447]  req@ffff9f14852d0a80 x1862022114705920/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767472 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767447.563521:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0a80 x1862022114705920/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767472 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767447.563526:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767447.563528:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767447.563531:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114705920:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767447.563535:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114706048:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767447.563545:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767447.563551:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767447/real 1775767447]  req@ffff9f14852d0700 x1862022114706048/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767472 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767447.563592:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0700 x1862022114706048/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767472 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767447.563598:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767447.563600:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767447.563602:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114706048:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775767447.818774:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775767448.586637:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775767448.842705:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775767449.610669:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:0.0:1775767449.866715:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775767450.634717:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767450.890682:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775767451.658689:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767451.914695:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775767452.682657:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:1.0:1775767452.682682:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767452.682689:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1775767452.682694:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767452.682703:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4830
00000100:00080000:1.0:1775767452.682715:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767452.682722:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775767452.682761:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff100 x1862022114706176/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767452.682788:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767452.682791:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775767452.682792:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767452.682795:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4830
00000100:00080000:1.0:1775767452.682798:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767452.682801:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767452.682808:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767452.682818:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fca80 x1862022114706304/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767452.682827:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1775767452.682835:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114706176:192.168.202.110@tcp:250:kworker.0
00000800:00000100:0.0:1775767452.682888:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:1.0:1775767452.682909:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775767452.682913:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767452.682922:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4830
00000100:00100000:0.0:1775767452.682923:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767452/real 1775767452]  req@ffff9f14ab4ff100 x1862022114706176/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767477 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767452.682932:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767452.682939:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775767452.682943:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff100 x1862022114706176/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767477 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:1.0:1775767452.682950:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00080000:0.0:1775767452.682953:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767452.682958:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767452.682963:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114706176:192.168.202.110@tcp:250:kworker.0
00000100:00100000:1.0:1775767452.682981:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ffb80 x1862022114706432/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775767452.682986:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114706304:192.168.202.110@tcp:38:kworker.0
00000100:00080000:1.0:1775767452.683021:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767452.683024:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775767452.683026:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767452.683033:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4830
00000800:00000100:0.0:1775767452.683036:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:1.0:1775767452.683043:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767452.683049:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775767452.683054:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767452/real 1775767452]  req@ffff9f14ab4fca80 x1862022114706304/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767477 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:1.0:1775767452.683060:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775767452.683080:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fca80 x1862022114706304/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767477 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767452.683084:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fed80 x1862022114706560/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767452.683141:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767452.683142:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767452.683144:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1775767452.683150:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767452.683159:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00080000:1.0:1775767452.683159:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4830
00000100:00080000:1.0:1775767452.683166:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767452.683174:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775767452.683175:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114706304:192.168.202.110@tcp:38:kworker.0
00000008:00100000:1.0:1775767452.683180:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775767452.683192:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff800 x1862022114706688/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767452.683203:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1775767452.683204:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ffb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114706432:192.168.202.110@tcp:38:kworker.0
00000100:00080000:1.0:1775767452.683208:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775767452.683209:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767452.683218:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4830
00000100:00080000:1.0:1775767452.683220:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767452.683224:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767452.683227:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767452.683238:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fca80 x1862022114706816/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767452.683244:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000800:00000100:0.0:1775767452.683253:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:1.0:1775767452.683257:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775767452.683259:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767452.683265:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4830
00000100:00080000:1.0:1775767452.683269:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767452.683274:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775767452.683276:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767452/real 1775767452]  req@ffff9f14ab4ffb80 x1862022114706432/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767477 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:1.0:1775767452.683280:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767452.683297:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff100 x1862022114706944/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775767452.683301:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ffb80 x1862022114706432/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767477 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767452.683340:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767452.683342:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767452.683346:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775767452.683351:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767452.683352:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00080000:1.0:1775767452.683364:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4830
00000100:00100000:0.0:1775767452.683365:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ffb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114706432:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767452.683378:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114706560:192.168.202.110@tcp:8:kworker.0
00000100:00080000:1.0:1775767452.683382:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767452.683387:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767452.683394:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:0.0:1775767452.683404:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767452.683405:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdc00 x1862022114707072/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775767452.683430:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767452/real 1775767452]  req@ffff9f14ab4fed80 x1862022114706560/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767477 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767452.683453:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fed80 x1862022114706560/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767477 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767452.683460:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767452.683462:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767452.683465:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114706560:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775767452.683485:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114706688:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767452.683500:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767452.683507:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767452/real 1775767452]  req@ffff9f14ab4ff800 x1862022114706688/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767477 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767452.683515:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff800 x1862022114706688/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767477 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767452.683520:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767452.683522:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767452.683523:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114706688:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775767452.683528:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114706816:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767452.683539:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767452.683544:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767452/real 1775767452]  req@ffff9f14ab4fca80 x1862022114706816/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767477 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767452.683552:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fca80 x1862022114706816/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767477 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767452.683586:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767452.683589:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767452.683591:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114706816:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767452.683595:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114706944:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767452.683608:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767452.683634:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767452/real 1775767452]  req@ffff9f14ab4ff100 x1862022114706944/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767477 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767452.683646:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff100 x1862022114706944/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767477 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767452.683654:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767452.683658:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767452.683661:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114706944:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767452.683668:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114707072:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767452.683690:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767452.683701:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767452/real 1775767452]  req@ffff9f14ab4fdc00 x1862022114707072/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767477 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767452.683709:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdc00 x1862022114707072/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767477 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767452.683715:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767452.683717:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767452.683719:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114707072:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775767452.938730:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775767453.706637:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767453.962766:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775767454.730663:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767454.986728:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767455.754676:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775767456.010715:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767456.778651:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:0.0:1775767457.034695:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767457.802633:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:0.0:1775767457.802680:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767457.802686:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1775767457.802689:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767457.802697:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4835
00000100:00080000:0.0:1775767457.802707:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767457.802713:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775767457.802756:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d2a00 x1862022114707200/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767457.802792:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767457.802797:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775767457.802799:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767457.802803:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4835
00000100:00080000:0.0:1775767457.802808:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767457.802812:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767457.802821:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767457.802832:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3800 x1862022114707328/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767457.802840:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767457.802844:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00100000:3.0:1775767457.802844:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114707200:192.168.202.110@tcp:250:kworker.0
00000100:00080000:0.0:1775767457.802847:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767457.802855:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4835
00000100:00080000:0.0:1775767457.802857:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767457.802861:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767457.802864:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767457.802872:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d1500 x1862022114707456/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767457.802878:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767457.802881:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775767457.802882:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767457.802884:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4835
00000100:00080000:0.0:1775767457.802886:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767457.802889:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767457.802892:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775767457.802899:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0000 x1862022114707584/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:3.0:1775767457.802905:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:0.0:1775767457.802911:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767457.802913:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1775767457.802915:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767457.802919:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4835
00000100:00080000:0.0:1775767457.802925:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767457.802927:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767457.802930:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775767457.802940:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d2300 x1862022114707712/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775767457.802945:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767457/real 1775767457]  req@ffff9f14a04d2a00 x1862022114707200/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767482 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767457.802951:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767457.802959:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775767457.802962:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767457.802966:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4835
00000100:00080000:0.0:1775767457.802970:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:3.0:1775767457.802973:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d2a00 x1862022114707200/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767482 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767457.802975:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767457.802981:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00080000:3.0:1775767457.802988:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00100000:0.0:1775767457.802993:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3100 x1862022114707840/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767457.802994:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767457.803002:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114707200:192.168.202.110@tcp:250:kworker.0
00000100:00080000:0.0:1775767457.803006:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767457.803009:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:0.0:1775767457.803011:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767457.803015:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4835
00000100:00080000:0.0:1775767457.803018:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767457.803020:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767457.803023:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767457.803027:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114707328:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767457.803031:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0a80 x1862022114707968/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767457.803040:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767457.803042:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775767457.803044:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767457.803046:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4835
00000100:00080000:0.0:1775767457.803050:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000800:00000100:3.0:1775767457.803054:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:0.0:1775767457.803058:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767457.803063:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775767457.803069:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767457/real 1775767457]  req@ffff9f14a04d3800 x1862022114707328/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767482 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767457.803072:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0e00 x1862022114708096/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775767457.803095:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3800 x1862022114707328/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767482 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767457.803103:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767457.803106:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767457.803110:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114707328:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767457.803118:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114707456:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775767457.803138:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767457.803148:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767457/real 1775767457]  req@ffff9f14a04d1500 x1862022114707456/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767482 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767457.803162:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1500 x1862022114707456/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767482 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767457.803171:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767457.803174:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767457.803178:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114707456:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767457.803184:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114707584:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775767457.803200:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767457.803208:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767457/real 1775767457]  req@ffff9f14a04d0000 x1862022114707584/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767482 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767457.803216:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0000 x1862022114707584/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767482 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767457.803221:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767457.803223:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767457.803226:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114707584:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775767457.803230:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114707712:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775767457.803250:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767457.803256:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767457/real 1775767457]  req@ffff9f14a04d2300 x1862022114707712/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767482 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767457.803263:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d2300 x1862022114707712/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767482 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767457.803269:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767457.803271:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767457.803273:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114707712:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775767457.803276:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114707840:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775767457.803285:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767457.803291:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767457/real 1775767457]  req@ffff9f14a04d3100 x1862022114707840/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767482 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767457.803298:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3100 x1862022114707840/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767482 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767457.803303:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767457.803305:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767457.803306:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114707840:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767457.803319:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114707968:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775767457.803329:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767457.803335:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767457/real 1775767457]  req@ffff9f14a04d0a80 x1862022114707968/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767482 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767457.803342:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0a80 x1862022114707968/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767482 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767457.803347:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767457.803349:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767457.803351:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114707968:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767457.803354:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114708096:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775767457.803377:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767457.803387:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767457/real 1775767457]  req@ffff9f14a04d0e00 x1862022114708096/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767482 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767457.803396:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0e00 x1862022114708096/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767482 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767457.803401:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767457.803403:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767457.803405:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114708096:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775767458.058698:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767458.826668:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767459.082720:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767459.850667:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767460.106759:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767460.874674:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767461.130714:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767461.898659:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775767462.154698:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1775767462.922623:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:2.0:1775767462.922623:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:0.0:1775767462.922632:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1775767462.922635:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767462.922643:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4840
00000100:00080000:0.0:1775767462.922653:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767462.922659:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775767462.922690:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3b80 x1862022114708224/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767462.922710:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767462.922713:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775767462.922714:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767462.922717:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4840
00000100:00080000:0.0:1775767462.922720:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767462.922722:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767462.922729:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767462.922738:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d1180 x1862022114708352/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767462.922747:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:3.0:1775767462.922748:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114708224:192.168.202.110@tcp:250:kworker.0
00000100:00080000:0.0:1775767462.922750:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:0.0:1775767462.922753:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767462.922757:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4840
00000100:00080000:0.0:1775767462.922759:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767462.922764:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767462.922768:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767462.922775:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3480 x1862022114708480/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767462.922781:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767462.922783:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775767462.922785:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767462.922787:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4840
00000100:00080000:0.0:1775767462.922789:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767462.922792:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767462.922795:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775767462.922801:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0380 x1862022114708608/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767462.922807:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767462.922809:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1775767462.922810:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767462.922813:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4840
00000100:00080000:0.0:1775767462.922816:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767462.922818:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767462.922822:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775767462.922829:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d1f80 x1862022114708736/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767462.922841:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767462.922843:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775767462.922844:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767462.922847:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4840
00000100:00080000:0.0:1775767462.922849:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:3.0:1775767462.922852:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114708352:192.168.202.110@tcp:38:kworker.0
00000100:00080000:0.0:1775767462.922855:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767462.922864:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767462.922880:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0e00 x1862022114708864/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775767462.922885:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114708480:192.168.202.110@tcp:38:kworker.0
00000100:00080000:0.0:1775767462.922891:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767462.922894:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:0.0:1775767462.922896:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767462.922899:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4840
00000100:00080000:0.0:1775767462.922903:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767462.922907:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775767462.922907:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114708608:192.168.202.110@tcp:8:kworker.0
00000008:00100000:0.0:1775767462.922915:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767462.922927:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0a80 x1862022114708992/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775767462.922936:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114708736:192.168.202.110@tcp:8:kworker.0
00000100:00080000:0.0:1775767462.922938:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767462.922944:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775767462.922946:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767462.922953:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4840
00000100:00080000:0.0:1775767462.922958:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:3.0:1775767462.922959:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114708864:192.168.202.110@tcp:38:kworker.0
00000100:00080000:0.0:1775767462.922967:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767462.922974:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775767462.922978:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114708992:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767462.922993:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3100 x1862022114709120/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775767462.923052:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114709120:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775767463.178754:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767463.946734:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:0.0:1775767464.202723:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767464.970710:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767465.226710:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767465.994756:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000400:00000100:1.0:1775767465.994849:0:2391:0:(lib-socket.c:454:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.202.110:988
00000400:00000100:1.0:1775767465.994884:0:2391:0:(acceptor.c:111:lnet_connect_console_error()) Connection to 192.168.202.110@tcp at host 192.168.202.110 was unreachable: the network or that node may be down, or Lustre may be misconfigured.
00000800:00000100:1.0:1775767465.994898:0:2391:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:1.0:1775767465.994935:0:2391:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:1.0:1775767465.994947:0:2391:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:1.0:1775767465.994959:0:2391:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:1.0:1775767465.994970:0:2391:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000100:00100000:3.0:1775767465.994975:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767462/real 1775767465]  req@ffff9f14a04d1180 x1862022114708352/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767487 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000800:00000100:1.0:1775767465.994976:0:2391:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:1.0:1775767465.994987:0:2391:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:1.0:1775767465.994998:0:2391:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000100:00100000:3.0:1775767465.995019:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1180 x1862022114708352/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767487 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767465.995032:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767465.995037:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767465.995042:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114708352:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767465.995050:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767462/real 1775767465]  req@ffff9f14a04d3480 x1862022114708480/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767487 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767465.995062:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3480 x1862022114708480/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767487 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767465.995071:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767465.995075:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767465.995078:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114708480:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767465.995085:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767462/real 1775767465]  req@ffff9f14a04d0380 x1862022114708608/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767487 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767465.995103:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0380 x1862022114708608/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767487 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767465.995110:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767465.995113:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767465.995116:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114708608:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775767465.995128:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767462/real 1775767465]  req@ffff9f14a04d1f80 x1862022114708736/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767487 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767465.995139:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1f80 x1862022114708736/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767487 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767465.995147:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767465.995150:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767465.995153:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114708736:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775767465.995158:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767462/real 1775767465]  req@ffff9f14a04d3b80 x1862022114708224/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767487 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767465.995169:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3b80 x1862022114708224/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767487 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767465.995176:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767465.995183:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767465.995187:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114708224:192.168.202.110@tcp:250:kworker.0
00000100:00100000:3.0:1775767465.995221:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767462/real 1775767465]  req@ffff9f14a04d3100 x1862022114709120/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767487 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767465.995234:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3100 x1862022114709120/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767487 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767465.995242:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767465.995246:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767465.995249:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114709120:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775767465.995273:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767462/real 1775767465]  req@ffff9f14a04d0e00 x1862022114708864/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767487 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767465.995303:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0e00 x1862022114708864/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767487 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767465.995312:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767465.995315:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767465.995318:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114708864:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767465.995326:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767462/real 1775767465]  req@ffff9f14a04d0a80 x1862022114708992/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767487 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767465.995338:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0a80 x1862022114708992/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767487 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767465.995345:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767465.995349:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767465.995352:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114708992:192.168.202.110@tcp:38:kworker.0
00010000:00010000:2.0:1775767466.250717:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767467.018710:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767467.274718:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767468.042697:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:1.0:1775767468.042743:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767468.042748:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1775767468.042751:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767468.042758:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4845
00000100:00080000:1.0:1775767468.042767:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767468.042772:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775767468.042801:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fd880 x1862022114709248/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767468.042831:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767468.042834:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775767468.042836:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767468.042838:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4845
00000100:00080000:1.0:1775767468.042841:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767468.042843:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767468.042851:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767468.042859:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fc380 x1862022114709376/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767468.042866:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767468.042870:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775767468.042871:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767468.042874:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4845
00000100:00080000:1.0:1775767468.042875:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767468.042878:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767468.042881:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767468.042887:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdf80 x1862022114709504/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767468.042894:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767468.042897:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775767468.042899:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767468.042901:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4845
00000100:00080000:1.0:1775767468.042905:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767468.042913:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767468.042917:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775767468.042926:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fe300 x1862022114709632/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775767468.042935:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114709248:192.168.202.110@tcp:250:kworker.0
00000100:00080000:1.0:1775767468.042937:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767468.042946:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1775767468.042951:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767468.042960:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4845
00000100:00080000:1.0:1775767468.042962:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767468.042965:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767468.042967:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775767468.042976:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fea00 x1862022114709760/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767468.042985:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767468.042988:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775767468.042990:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767468.042995:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4845
00000100:00080000:1.0:1775767468.042999:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767468.043005:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767468.043011:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767468.043023:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fce00 x1862022114709888/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767468.043039:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767468.043044:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775767468.043046:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767468.043051:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4845
00000100:00080000:1.0:1775767468.043056:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767468.043059:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767468.043065:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000800:00000100:3.0:1775767468.043074:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767468.043087:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdc00 x1862022114710016/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767468.043101:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767468.043105:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775767468.043107:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767468.043111:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4845
00000100:00080000:1.0:1775767468.043116:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767468.043120:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767468.043128:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775767468.043143:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff100 x1862022114710144/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775767468.043155:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767468/real 1775767468]  req@ffff9f14ab4fd880 x1862022114709248/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767493 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767468.043188:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fd880 x1862022114709248/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767493 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767468.043203:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767468.043209:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767468.043214:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114709248:192.168.202.110@tcp:250:kworker.0
00000100:00100000:3.0:1775767468.043496:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114709376:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775767468.043662:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767468.043679:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767468/real 1775767468]  req@ffff9f14ab4fc380 x1862022114709376/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767493 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767468.043696:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fc380 x1862022114709376/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767493 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767468.043705:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767468.043710:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767468.043713:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114709376:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767468.043720:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114709504:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775767468.043743:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767468.043755:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767468/real 1775767468]  req@ffff9f14ab4fdf80 x1862022114709504/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767493 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767468.043775:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdf80 x1862022114709504/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767493 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767468.043785:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767468.043789:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767468.043792:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114709504:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767468.043829:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114709632:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775767468.043852:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767468.043864:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767468/real 1775767468]  req@ffff9f14ab4fe300 x1862022114709632/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767493 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767468.043875:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fe300 x1862022114709632/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767493 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767468.043883:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767468.043886:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767468.043890:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114709632:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775767468.043897:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114709760:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775767468.043914:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767468.043923:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767468/real 1775767468]  req@ffff9f14ab4fea00 x1862022114709760/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767493 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767468.043934:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fea00 x1862022114709760/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767493 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767468.043947:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767468.043950:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767468.043952:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114709760:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775767468.043958:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114709888:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775767468.043977:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767468.043987:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767468/real 1775767468]  req@ffff9f14ab4fce00 x1862022114709888/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767493 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767468.044000:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fce00 x1862022114709888/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767493 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767468.044008:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767468.044012:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767468.044015:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114709888:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767468.044021:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114710016:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775767468.044041:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767468.044053:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767468/real 1775767468]  req@ffff9f14ab4fdc00 x1862022114710016/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767493 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767468.044064:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdc00 x1862022114710016/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767493 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767468.044072:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767468.044075:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767468.044079:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114710016:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767468.044085:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114710144:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775767468.044106:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767468.044116:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767468/real 1775767468]  req@ffff9f14ab4ff100 x1862022114710144/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767493 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767468.044128:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff100 x1862022114710144/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767493 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767468.044136:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767468.044139:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767468.044142:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114710144:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775767468.298789:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767469.066765:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767469.322767:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767470.090663:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767470.346749:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767471.114628:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:0.0:1775767471.370673:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767472.138653:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767472.394731:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767473.162625:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:1.0:1775767473.162714:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767473.162723:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1775767473.162728:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767473.162739:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4850
00000100:00080000:1.0:1775767473.162752:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767473.162759:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775767473.162816:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fca80 x1862022114710272/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767473.162844:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767473.162846:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775767473.162848:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767473.162851:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4850
00000100:00080000:1.0:1775767473.162854:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767473.162856:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767473.162866:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767473.162876:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff800 x1862022114710400/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767473.162884:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767473.162888:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775767473.162890:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767473.162893:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4850
00000100:00080000:1.0:1775767473.162895:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767473.162898:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767473.162901:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767473.162911:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fed80 x1862022114710528/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775767473.162913:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114710272:192.168.202.110@tcp:250:kworker.0
00000100:00080000:1.0:1775767473.162937:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767473.162940:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775767473.162941:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767473.162944:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4850
00000100:00080000:1.0:1775767473.162946:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767473.162949:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767473.162953:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775767473.162961:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ffb80 x1862022114710656/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767473.162966:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767473.162968:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1775767473.162970:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767473.162973:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4850
00000100:00080000:1.0:1775767473.162976:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767473.162980:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767473.162983:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775767473.162990:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff100 x1862022114710784/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767473.162996:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767473.162999:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775767473.163001:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767473.163006:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4850
00000800:00000100:3.0:1775767473.163011:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:1.0:1775767473.163013:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767473.163020:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767473.163027:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767473.163046:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdc00 x1862022114710912/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767473.163058:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767473.163064:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00100000:3.0:1775767473.163064:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767473/real 1775767473]  req@ffff9f14ab4fca80 x1862022114710272/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767498 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767473.163069:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767473.163077:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4850
00000100:00080000:1.0:1775767473.163084:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767473.163095:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767473.163105:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767473.163118:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fce00 x1862022114711040/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775767473.163119:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fca80 x1862022114710272/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767498 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767473.163151:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767473.163159:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775767473.163164:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767473.163173:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4850
00000100:00080000:1.0:1775767473.163178:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767473.163180:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00080000:3.0:1775767473.163181:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000008:00100000:1.0:1775767473.163184:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00080000:3.0:1775767473.163194:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767473.163195:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fea00 x1862022114711168/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775767473.163209:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114710272:192.168.202.110@tcp:250:kworker.0
00000100:00100000:3.0:1775767473.163239:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114710400:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775767473.163270:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767473.163285:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767473/real 1775767473]  req@ffff9f14ab4ff800 x1862022114710400/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767498 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767473.163301:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff800 x1862022114710400/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767498 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767473.163311:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767473.163315:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767473.163318:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114710400:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767473.163342:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114710528:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775767473.163366:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767473.163378:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767473/real 1775767473]  req@ffff9f14ab4fed80 x1862022114710528/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767498 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767473.163392:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fed80 x1862022114710528/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767498 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767473.163400:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767473.163404:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767473.163408:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114710528:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767473.163416:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ffb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114710656:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775767473.163445:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767473.163453:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767473/real 1775767473]  req@ffff9f14ab4ffb80 x1862022114710656/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767498 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767473.163465:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ffb80 x1862022114710656/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767498 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767473.163474:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767473.163478:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767473.163481:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ffb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114710656:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775767473.163488:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114710784:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775767473.163509:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767473.163521:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767473/real 1775767473]  req@ffff9f14ab4ff100 x1862022114710784/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767498 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767473.163534:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff100 x1862022114710784/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767498 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767473.163543:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767473.163546:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767473.163550:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114710784:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775767473.163594:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114710912:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775767473.163611:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767473.163619:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767473/real 1775767473]  req@ffff9f14ab4fdc00 x1862022114710912/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767498 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767473.163627:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdc00 x1862022114710912/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767498 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767473.163633:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767473.163635:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767473.163637:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114710912:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767473.163642:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114711040:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775767473.163652:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767473.163657:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767473/real 1775767473]  req@ffff9f14ab4fce00 x1862022114711040/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767498 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767473.163664:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fce00 x1862022114711040/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767498 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767473.163669:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767473.163671:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767473.163673:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114711040:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767473.163677:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114711168:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775767473.163686:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767473.163692:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767473/real 1775767473]  req@ffff9f14ab4fea00 x1862022114711168/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767498 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767473.163700:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fea00 x1862022114711168/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767498 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767473.163708:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767473.163711:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767473.163715:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114711168:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775767473.418764:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767474.186672:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775767474.442722:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767475.210609:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767475.466688:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767476.234662:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767476.490736:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767477.258687:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767477.514707:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767478.282674:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:0.0:1775767478.282710:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767478.282716:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1775767478.282720:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767478.282727:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4855
00000100:00080000:0.0:1775767478.282737:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767478.282742:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775767478.282776:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d2300 x1862022114711296/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767478.282801:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767478.282805:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775767478.282807:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767478.282811:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4855
00000100:00080000:0.0:1775767478.282814:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767478.282818:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767478.282827:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767478.282843:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0000 x1862022114711424/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775767478.282852:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114711296:192.168.202.110@tcp:250:kworker.0
00000100:00080000:0.0:1775767478.282860:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767478.282864:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:0.0:1775767478.282865:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767478.282869:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4855
00000100:00080000:0.0:1775767478.282872:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767478.282875:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767478.282880:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767478.282889:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d1500 x1862022114711552/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767478.282897:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767478.282900:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775767478.282902:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767478.282905:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4855
00000100:00080000:0.0:1775767478.282909:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000800:00000100:3.0:1775767478.282909:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:0.0:1775767478.282912:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767478.282917:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775767478.282926:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3800 x1862022114711680/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767478.282934:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767478.282937:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1775767478.282939:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767478.282942:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4855
00000100:00080000:0.0:1775767478.282946:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767478.282949:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767478.282954:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775767478.282954:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767478/real 1775767478]  req@ffff9f14a04d2300 x1862022114711296/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767503 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767478.282962:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d2a00 x1862022114711808/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767478.282971:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:3.0:1775767478.282973:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d2300 x1862022114711296/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767503 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767478.282974:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775767478.282976:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767478.282979:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4855
00000100:00080000:0.0:1775767478.282982:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767478.282983:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767478.282986:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00080000:3.0:1775767478.282987:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000008:00100000:0.0:1775767478.282990:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767478.282991:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114711296:192.168.202.110@tcp:250:kworker.0
00000100:00100000:0.0:1775767478.282998:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0a80 x1862022114711936/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767478.283005:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767478.283008:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:0.0:1775767478.283010:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:3.0:1775767478.283012:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114711424:192.168.202.110@tcp:38:kworker.0
00000100:00080000:0.0:1775767478.283014:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4855
00000100:00080000:0.0:1775767478.283018:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767478.283021:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767478.283026:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000800:00000100:3.0:1775767478.283032:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767478.283034:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0e00 x1862022114712064/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767478.283050:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767478.283054:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00100000:3.0:1775767478.283054:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767478/real 1775767478]  req@ffff9f14a04d0000 x1862022114711424/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767503 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767478.283056:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767478.283060:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4855
00000100:00080000:0.0:1775767478.283064:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:3.0:1775767478.283067:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0000 x1862022114711424/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767503 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767478.283068:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767478.283073:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00080000:3.0:1775767478.283076:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767478.283079:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767478.283082:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3100 x1862022114712192/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775767478.283082:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114711424:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767478.283089:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114711552:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775767478.283108:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767478.283119:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767478/real 1775767478]  req@ffff9f14a04d1500 x1862022114711552/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767503 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767478.283132:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1500 x1862022114711552/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767503 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767478.283142:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767478.283144:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767478.283146:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114711552:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767478.283150:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114711680:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775767478.283181:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767478.283191:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767478/real 1775767478]  req@ffff9f14a04d3800 x1862022114711680/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767503 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767478.283199:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3800 x1862022114711680/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767503 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767478.283204:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767478.283206:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767478.283208:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114711680:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775767478.283212:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114711808:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775767478.283228:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767478.283239:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767478/real 1775767478]  req@ffff9f14a04d2a00 x1862022114711808/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767503 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767478.283252:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d2a00 x1862022114711808/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767503 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767478.283260:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767478.283264:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767478.283268:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114711808:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775767478.283274:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114711936:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775767478.283293:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767478.283304:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767478/real 1775767478]  req@ffff9f14a04d0a80 x1862022114711936/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767503 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767478.283315:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0a80 x1862022114711936/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767503 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767478.283322:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767478.283325:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767478.283328:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114711936:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767478.283346:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114712064:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775767478.283363:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767478.283372:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767478/real 1775767478]  req@ffff9f14a04d0e00 x1862022114712064/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767503 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767478.283383:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0e00 x1862022114712064/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767503 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767478.283390:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767478.283393:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767478.283396:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114712064:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767478.283401:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114712192:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775767478.283417:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767478.283426:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767478/real 1775767478]  req@ffff9f14a04d3100 x1862022114712192/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767503 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767478.283437:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3100 x1862022114712192/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767503 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767478.283444:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767478.283447:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767478.283450:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114712192:192.168.202.110@tcp:8:kworker.0
00010000:00010000:0.0:1775767478.538793:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767479.306679:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767479.562711:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767480.330707:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767480.586754:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767481.354743:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775767481.610727:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767482.378661:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767482.634761:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1775767483.402627:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767483.402633:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1775767483.402638:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767483.402645:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4860
00000100:00080000:1.0:1775767483.402655:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767483.402662:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775767483.402693:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fe300 x1862022114712320/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775767483.402703:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:1.0:1775767483.402718:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767483.402724:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775767483.402726:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767483.402730:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4860
00000100:00080000:1.0:1775767483.402733:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767483.402737:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767483.402745:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767483.402756:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdf80 x1862022114712448/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775767483.402764:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114712320:192.168.202.110@tcp:250:kworker.0
00000100:00080000:1.0:1775767483.402773:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767483.402777:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775767483.402779:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767483.402783:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4860
00000100:00080000:1.0:1775767483.402794:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767483.402798:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767483.402802:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767483.402815:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fc380 x1862022114712576/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767483.402825:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767483.402827:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775767483.402830:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767483.402835:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4860
00000100:00080000:1.0:1775767483.402846:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767483.402853:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000800:00000100:2.0:1775767483.402860:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000008:00100000:1.0:1775767483.402861:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775767483.402880:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fd880 x1862022114712704/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767483.402892:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767483.402896:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1775767483.402898:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767483.402904:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4860
00000100:00080000:1.0:1775767483.402911:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767483.402916:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775767483.402922:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767483/real 1775767483]  req@ffff9f14ab4fe300 x1862022114712320/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767508 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:1.0:1775767483.402923:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775767483.402951:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fea00 x1862022114712832/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775767483.402959:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fe300 x1862022114712320/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767508 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767483.402975:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767483.402981:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775767483.402982:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767483.402984:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767483.402994:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:1.0:1775767483.402996:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4860
00000100:00080000:1.0:1775767483.403005:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:2.0:1775767483.403006:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114712320:192.168.202.110@tcp:250:kworker.0
00000100:00080000:1.0:1775767483.403017:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767483.403025:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767483.403040:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fce00 x1862022114712960/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775767483.403046:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114712448:192.168.202.110@tcp:38:kworker.0
00000100:00080000:1.0:1775767483.403080:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767483.403084:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775767483.403086:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000800:00000100:2.0:1775767483.403094:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:1.0:1775767483.403100:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4860
00000100:00080000:1.0:1775767483.403106:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767483.403112:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767483.403130:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775767483.403134:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767483/real 1775767483]  req@ffff9f14ab4fdf80 x1862022114712448/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767508 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767483.403148:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdc00 x1862022114713088/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775767483.403153:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdf80 x1862022114712448/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767508 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767483.403215:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767483.403217:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767483.403222:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00080000:1.0:1775767483.403229:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775767483.403233:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:2.0:1775767483.403234:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114712448:192.168.202.110@tcp:38:kworker.0
00000100:00080000:1.0:1775767483.403254:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4860
00000100:00080000:1.0:1775767483.403257:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:2.0:1775767483.403259:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114712576:192.168.202.110@tcp:38:kworker.0
00000100:00080000:1.0:1775767483.403263:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767483.403268:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775767483.403285:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff100 x1862022114713216/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:2.0:1775767483.403308:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767483.403324:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767483/real 1775767483]  req@ffff9f14ab4fc380 x1862022114712576/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767508 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767483.403338:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fc380 x1862022114712576/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767508 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767483.403353:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767483.403357:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767483.403361:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114712576:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767483.403369:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114712704:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775767483.403392:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767483.403404:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767483/real 1775767483]  req@ffff9f14ab4fd880 x1862022114712704/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767508 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767483.403416:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fd880 x1862022114712704/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767508 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767483.403424:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767483.403428:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767483.403431:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114712704:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775767483.403437:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114712832:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775767483.403457:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767483.403468:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767483/real 1775767483]  req@ffff9f14ab4fea00 x1862022114712832/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767508 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767483.403481:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fea00 x1862022114712832/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767508 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767483.403491:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767483.403494:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767483.403498:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114712832:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775767483.403519:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114712960:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775767483.403536:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767483.403545:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767483/real 1775767483]  req@ffff9f14ab4fce00 x1862022114712960/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767508 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767483.403598:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fce00 x1862022114712960/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767508 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767483.403615:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767483.403620:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767483.403625:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114712960:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767483.403632:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114713088:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775767483.403652:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767483.403661:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767483/real 1775767483]  req@ffff9f14ab4fdc00 x1862022114713088/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767508 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767483.403669:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdc00 x1862022114713088/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767508 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767483.403675:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767483.403677:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767483.403679:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114713088:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767483.403683:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114713216:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775767483.403694:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767483.403701:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767483/real 1775767483]  req@ffff9f14ab4ff100 x1862022114713216/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767508 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767483.403708:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff100 x1862022114713216/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767508 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767483.403713:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767483.403715:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767483.403717:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114713216:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775767483.658822:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767484.426711:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767484.682734:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767485.450639:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:0.0:1775767485.706712:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767486.474696:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767486.730809:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767487.498688:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767487.754764:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767488.522668:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:0.0:1775767488.522703:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767488.522709:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1775767488.522714:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767488.522724:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4866
00000100:00080000:0.0:1775767488.522737:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767488.522744:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775767488.522790:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3b80 x1862022114713344/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767488.522816:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767488.522819:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775767488.522820:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767488.522824:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4866
00000100:00080000:0.0:1775767488.522827:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767488.522831:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767488.522841:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775767488.522846:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114713344:192.168.202.110@tcp:250:kworker.0
00000100:00100000:0.0:1775767488.522853:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d1f80 x1862022114713472/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767488.522861:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767488.522864:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:0.0:1775767488.522865:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767488.522868:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4866
00000100:00080000:0.0:1775767488.522870:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767488.522873:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767488.522876:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767488.522884:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0380 x1862022114713600/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767488.522890:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767488.522893:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775767488.522894:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767488.522896:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4866
00000100:00080000:0.0:1775767488.522898:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767488.522900:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767488.522903:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775767488.522909:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3480 x1862022114713728/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767488.522919:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767488.522922:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1775767488.522926:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000800:00000100:2.0:1775767488.522927:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:0.0:1775767488.522932:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4866
00000100:00080000:0.0:1775767488.522935:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767488.522937:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767488.522941:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775767488.522951:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d1180 x1862022114713856/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767488.522965:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767488.522968:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775767488.522969:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767488.522972:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4866
00000100:00080000:0.0:1775767488.522975:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767488.522979:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775767488.522982:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767488/real 1775767488]  req@ffff9f14a04d3b80 x1862022114713344/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767513 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:0.0:1775767488.522984:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767488.522998:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3100 x1862022114713984/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775767488.523005:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3b80 x1862022114713344/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767513 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767488.523007:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767488.523014:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:0.0:1775767488.523018:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767488.523026:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4866
00000100:00080000:2.0:1775767488.523027:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767488.523030:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767488.523034:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:0.0:1775767488.523035:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767488.523041:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775767488.523042:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114713344:192.168.202.110@tcp:250:kworker.0
00000100:00100000:0.0:1775767488.523051:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0e00 x1862022114714112/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767488.523061:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767488.523064:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775767488.523080:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:2.0:1775767488.523081:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114713472:192.168.202.110@tcp:38:kworker.0
00000100:00080000:0.0:1775767488.523089:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4866
00000100:00080000:0.0:1775767488.523093:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767488.523096:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767488.523101:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:2.0:1775767488.523111:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767488.523120:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0a80 x1862022114714240/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775767488.523123:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767488/real 1775767488]  req@ffff9f14a04d1f80 x1862022114713472/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767513 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767488.523145:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1f80 x1862022114713472/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767513 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767488.523155:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767488.523159:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767488.523162:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114713472:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767488.523167:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114713600:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775767488.523186:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767488.523197:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767488/real 1775767488]  req@ffff9f14a04d0380 x1862022114713600/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767513 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767488.523210:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0380 x1862022114713600/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767513 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767488.523219:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767488.523223:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767488.523227:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114713600:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767488.523233:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114713728:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775767488.523254:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767488.523262:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767488/real 1775767488]  req@ffff9f14a04d3480 x1862022114713728/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767513 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767488.523271:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3480 x1862022114713728/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767513 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767488.523276:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767488.523278:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767488.523280:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114713728:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775767488.523284:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114713856:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775767488.523294:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767488.523299:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767488/real 1775767488]  req@ffff9f14a04d1180 x1862022114713856/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767513 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767488.523307:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1180 x1862022114713856/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767513 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767488.523312:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767488.523314:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767488.523315:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114713856:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775767488.523319:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114713984:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775767488.523328:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767488.523334:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767488/real 1775767488]  req@ffff9f14a04d3100 x1862022114713984/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767513 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767488.523341:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3100 x1862022114713984/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767513 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767488.523351:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767488.523353:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767488.523355:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114713984:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767488.523358:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114714112:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775767488.523368:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767488.523373:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767488/real 1775767488]  req@ffff9f14a04d0e00 x1862022114714112/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767513 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767488.523379:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0e00 x1862022114714112/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767513 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767488.523384:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767488.523387:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767488.523390:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114714112:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767488.523413:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114714240:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775767488.523435:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767488.523447:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767488/real 1775767488]  req@ffff9f14a04d0a80 x1862022114714240/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767513 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767488.523458:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0a80 x1862022114714240/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767513 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767488.523464:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767488.523466:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767488.523469:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114714240:192.168.202.110@tcp:8:kworker.0
00010000:00010000:1.0:1775767488.778777:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767489.546697:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775767489.802763:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767490.570672:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767490.826739:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767491.594712:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775767491.850734:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767492.618630:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:0.0:1775767492.874687:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767493.642653:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775767493.642680:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767493.642688:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775767493.642693:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767493.642703:0:8:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4871
00000100:00080000:3.0:1775767493.642714:0:8:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767493.642722:0:8:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775767493.642758:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744700 x1862022114714368/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767493.642781:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767493.642784:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775767493.642786:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767493.642790:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4871
00000100:00080000:3.0:1775767493.642794:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767493.642797:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767493.642806:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775767493.642809:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114714368:192.168.202.110@tcp:250:kworker.0
00000100:00100000:3.0:1775767493.642820:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747100 x1862022114714496/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767493.642829:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767493.642831:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775767493.642833:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767493.642835:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4871
00000100:00080000:3.0:1775767493.642838:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767493.642841:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767493.642845:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767493.642852:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745180 x1862022114714624/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767493.642858:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767493.642860:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775767493.642861:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767493.642864:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4871
00000800:00000100:2.0:1775767493.642869:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775767493.642869:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767493.642875:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767493.642879:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775767493.642887:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745c00 x1862022114714752/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767493.642894:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767493.642896:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775767493.642898:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767493.642901:0:8:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4871
00000100:00080000:3.0:1775767493.642905:0:8:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00100000:2.0:1775767493.642910:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767493/real 1775767493]  req@ffff9f1485744700 x1862022114714368/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767518 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767493.642918:0:8:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767493.642923:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775767493.642927:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744700 x1862022114714368/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767518 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767493.642933:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746680 x1862022114714880/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767493.642942:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767493.642946:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767493.642949:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:3.0:1775767493.642950:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775767493.642953:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:2.0:1775767493.642955:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114714368:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775767493.642959:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4871
00000100:00080000:3.0:1775767493.642964:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767493.642967:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767493.642971:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767493.642976:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744000 x1862022114715008/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767493.642982:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767493.642985:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775767493.642986:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:2.0:1775767493.642989:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114714496:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775767493.642993:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4871
00000100:00080000:3.0:1775767493.642998:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767493.643000:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767493.643004:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767493.643014:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747800 x1862022114715136/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:2.0:1775767493.643035:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767493.643048:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767493/real 1775767493]  req@ffff9f1485747100 x1862022114714496/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767518 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767493.643072:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747100 x1862022114714496/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767518 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767493.643083:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767493.643083:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767493.643090:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775767493.643092:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00080000:3.0:1775767493.643094:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:2.0:1775767493.643098:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114714496:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775767493.643102:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4871
00000100:00080000:3.0:1775767493.643107:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:2.0:1775767493.643108:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114714624:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775767493.643115:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767493.643120:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:2.0:1775767493.643130:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767493.643133:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745f80 x1862022114715264/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775767493.643140:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767493/real 1775767493]  req@ffff9f1485745180 x1862022114714624/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767518 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767493.643158:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745180 x1862022114714624/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767518 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767493.643164:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767493.643166:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767493.643169:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114714624:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767493.643174:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114714752:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775767493.643185:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767493.643190:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767493/real 1775767493]  req@ffff9f1485745c00 x1862022114714752/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767518 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767493.643197:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745c00 x1862022114714752/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767518 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767493.643202:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767493.643204:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767493.643205:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114714752:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775767493.643209:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114714880:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775767493.643218:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767493.643223:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767493/real 1775767493]  req@ffff9f1485746680 x1862022114714880/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767518 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767493.643230:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746680 x1862022114714880/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767518 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767493.643235:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767493.643237:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767493.643238:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114714880:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775767493.643241:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114715008:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775767493.643251:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767493.643256:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767493/real 1775767493]  req@ffff9f1485744000 x1862022114715008/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767518 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767493.643262:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744000 x1862022114715008/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767518 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767493.643267:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767493.643269:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767493.643271:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114715008:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767493.643283:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114715136:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775767493.643292:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767493.643297:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767493/real 1775767493]  req@ffff9f1485747800 x1862022114715136/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767518 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767493.643309:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747800 x1862022114715136/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767518 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767493.643314:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767493.643316:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767493.643318:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114715136:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767493.643321:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114715264:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775767493.643330:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767493.643335:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767493/real 1775767493]  req@ffff9f1485745f80 x1862022114715264/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767518 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767493.643342:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745f80 x1862022114715264/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767518 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767493.643347:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767493.643349:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767493.643350:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114715264:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775767493.898728:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767494.666631:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767494.922759:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767495.690669:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775767495.946774:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775767496.714639:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775767496.970664:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775767497.738646:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767497.994753:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1775767498.762692:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:3.0:1775767498.762696:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:0.0:1775767498.762702:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1775767498.762706:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767498.762726:0:8:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4876
00000100:00080000:0.0:1775767498.762735:0:8:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767498.762742:0:8:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775767498.762775:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d2a00 x1862022114715392/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767498.762799:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767498.762802:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775767498.762803:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767498.762806:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4876
00000100:00080000:0.0:1775767498.762808:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767498.762810:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767498.762818:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767498.762827:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3800 x1862022114715520/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767498.762835:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767498.762839:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:0.0:1775767498.762841:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:2.0:1775767498.762841:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114715392:192.168.202.110@tcp:250:kworker.0
00000100:00080000:0.0:1775767498.762846:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4876
00000100:00080000:0.0:1775767498.762850:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767498.762853:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767498.762855:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767498.762862:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d1500 x1862022114715648/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767498.762868:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767498.762870:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775767498.762872:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767498.762874:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4876
00000100:00080000:0.0:1775767498.762876:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767498.762879:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767498.762883:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775767498.762901:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0000 x1862022114715776/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767498.762907:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767498.762909:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1775767498.762911:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767498.762913:0:8:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4876
00000100:00080000:0.0:1775767498.762915:0:8:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767498.762917:0:8:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767498.762920:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:2.0:1775767498.762925:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767498.762929:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d2300 x1862022114715904/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767498.762935:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767498.762937:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775767498.762939:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767498.762942:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4876
00000100:00080000:0.0:1775767498.762947:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767498.762950:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767498.762953:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767498.762960:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0a80 x1862022114716032/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767498.762980:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:2.0:1775767498.762985:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767498/real 1775767498]  req@ffff9f14a04d2a00 x1862022114715392/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767523 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767498.762986:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:0.0:1775767498.762989:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767498.762997:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4876
00000100:00080000:0.0:1775767498.763003:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767498.763007:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767498.763012:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775767498.763014:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d2a00 x1862022114715392/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767523 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767498.763022:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0e00 x1862022114716160/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767498.763023:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767498.763032:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:0.0:1775767498.763037:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:2.0:1775767498.763038:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114715392:192.168.202.110@tcp:250:kworker.0
00000100:00080000:0.0:1775767498.763041:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775767498.763045:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767498.763049:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4876
00000100:00080000:0.0:1775767498.763051:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767498.763053:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767498.763056:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775767498.763062:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3100 x1862022114716288/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775767498.763065:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114715520:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775767498.763093:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767498.763103:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767498/real 1775767498]  req@ffff9f14a04d3800 x1862022114715520/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767523 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767498.763111:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3800 x1862022114715520/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767523 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767498.763117:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767498.763119:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767498.763121:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114715520:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767498.763125:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114715648:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775767498.763134:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767498.763140:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767498/real 1775767498]  req@ffff9f14a04d1500 x1862022114715648/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767523 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767498.763147:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1500 x1862022114715648/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767523 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767498.763156:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767498.763158:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767498.763160:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114715648:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767498.763163:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114715776:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775767498.763173:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767498.763178:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767498/real 1775767498]  req@ffff9f14a04d0000 x1862022114715776/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767523 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767498.763184:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0000 x1862022114715776/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767523 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767498.763189:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767498.763191:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767498.763193:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114715776:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775767498.763196:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114715904:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775767498.763205:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767498.763209:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767498/real 1775767498]  req@ffff9f14a04d2300 x1862022114715904/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767523 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767498.763216:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d2300 x1862022114715904/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767523 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767498.763220:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767498.763222:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767498.763225:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114715904:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775767498.763228:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114716032:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775767498.763238:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767498.763243:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767498/real 1775767498]  req@ffff9f14a04d0a80 x1862022114716032/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767523 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767498.763249:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0a80 x1862022114716032/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767523 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767498.763255:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767498.763256:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767498.763258:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114716032:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767498.763261:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114716160:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775767498.763269:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767498.763274:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767498/real 1775767498]  req@ffff9f14a04d0e00 x1862022114716160/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767523 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767498.763281:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0e00 x1862022114716160/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767523 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767498.763285:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767498.763287:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767498.763289:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114716160:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767498.763310:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114716288:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775767498.763320:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767498.763326:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767498/real 1775767498]  req@ffff9f14a04d3100 x1862022114716288/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767523 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767498.763333:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3100 x1862022114716288/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767523 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767498.763338:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767498.763339:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767498.763341:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114716288:192.168.202.110@tcp:8:kworker.0
00010000:00010000:1.0:1775767499.018808:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775767499.786667:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:0.0:1775767500.042717:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775767500.810686:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767501.066715:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775767501.834683:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767502.090834:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775767502.858669:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775767503.114773:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1775767503.882704:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767503.882714:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775767503.882719:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775767503.882725:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775767503.882731:0:8:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4881
00000100:00080000:3.0:1775767503.882750:0:8:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767503.882760:0:8:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775767503.882808:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745500 x1862022114716416/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767503.882835:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767503.882838:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775767503.882840:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767503.882843:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4881
00000100:00080000:3.0:1775767503.882846:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767503.882848:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767503.882857:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767503.882866:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744e00 x1862022114716544/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767503.882874:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767503.882876:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775767503.882877:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767503.882880:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4881
00000100:00080000:3.0:1775767503.882882:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767503.882885:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767503.882888:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775767503.882895:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114716416:192.168.202.110@tcp:250:kworker.0
00000100:00100000:3.0:1775767503.882895:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744380 x1862022114716672/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767503.882913:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767503.882916:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775767503.882930:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767503.882934:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4881
00000100:00080000:3.0:1775767503.882939:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767503.882943:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767503.882950:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775767503.882964:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746a00 x1862022114716800/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767503.882974:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767503.882978:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775767503.882980:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767503.882983:0:8:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4881
00000100:00080000:3.0:1775767503.882986:0:8:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767503.882989:0:8:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767503.882993:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:2.0:1775767503.882998:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767503.883005:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744a80 x1862022114716928/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767503.883012:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767503.883015:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775767503.883016:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767503.883019:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4881
00000100:00080000:3.0:1775767503.883021:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767503.883025:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767503.883027:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767503.883035:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747480 x1862022114717056/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767503.883042:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767503.883044:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775767503.883046:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767503.883049:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4881
00000100:00080000:3.0:1775767503.883052:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:2.0:1775767503.883054:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767503/real 1775767503]  req@ffff9f1485745500 x1862022114716416/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767528 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767503.883057:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767503.883064:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767503.883077:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745880 x1862022114717184/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775767503.883081:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745500 x1862022114716416/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767528 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767503.883087:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767503.883094:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775767503.883097:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767503.883103:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4881
00000100:00080000:2.0:1775767503.883104:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767503.883105:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767503.883111:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00080000:2.0:1775767503.883114:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000008:00100000:3.0:1775767503.883116:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775767503.883119:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114716416:192.168.202.110@tcp:250:kworker.0
00000100:00100000:3.0:1775767503.883125:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746300 x1862022114717312/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775767503.883128:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114716544:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775767503.883155:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767503.883166:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767503/real 1775767503]  req@ffff9f1485744e00 x1862022114716544/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767528 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767503.883180:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744e00 x1862022114716544/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767528 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767503.883186:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767503.883188:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767503.883190:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114716544:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767503.883214:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114716672:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775767503.883225:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767503.883231:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767503/real 1775767503]  req@ffff9f1485744380 x1862022114716672/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767528 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767503.883239:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744380 x1862022114716672/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767528 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767503.883244:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767503.883246:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767503.883248:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114716672:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767503.883251:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114716800:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775767503.883261:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767503.883266:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767503/real 1775767503]  req@ffff9f1485746a00 x1862022114716800/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767528 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767503.883274:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746a00 x1862022114716800/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767528 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767503.883283:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767503.883285:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767503.883287:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114716800:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775767503.883290:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114716928:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775767503.883300:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767503.883305:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767503/real 1775767503]  req@ffff9f1485744a80 x1862022114716928/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767528 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767503.883311:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744a80 x1862022114716928/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767528 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767503.883316:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767503.883318:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767503.883320:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114716928:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775767503.883324:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114717056:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775767503.883334:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767503.883339:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767503/real 1775767503]  req@ffff9f1485747480 x1862022114717056/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767528 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767503.883346:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747480 x1862022114717056/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767528 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767503.883351:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767503.883353:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767503.883356:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114717056:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767503.883360:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114717184:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775767503.883370:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767503.883375:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767503/real 1775767503]  req@ffff9f1485745880 x1862022114717184/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767528 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767503.883382:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745880 x1862022114717184/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767528 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767503.883386:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767503.883388:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767503.883390:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114717184:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767503.883393:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114717312:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775767503.883402:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767503.883407:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767503/real 1775767503]  req@ffff9f1485746300 x1862022114717312/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767528 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767503.883413:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746300 x1862022114717312/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767528 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767503.883419:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767503.883421:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767503.883423:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114717312:192.168.202.110@tcp:8:kworker.0
00010000:00010000:1.0:1775767504.138743:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767504.906665:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767505.162708:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767505.930699:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775767506.186769:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767506.954671:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767507.210768:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767507.978661:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767508.234762:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1775767509.002671:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:2.0:1775767509.002675:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775767509.002684:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775767509.002701:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767509.002711:0:8:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4886
00000100:00080000:3.0:1775767509.002723:0:8:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767509.002729:0:8:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775767509.002775:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747b80 x1862022114717440/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767509.002800:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767509.002804:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775767509.002806:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767509.002811:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4886
00000100:00080000:3.0:1775767509.002815:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767509.002819:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767509.002840:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767509.002856:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746300 x1862022114717568/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767509.002867:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767509.002881:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775767509.002883:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767509.002886:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4886
00000100:00080000:3.0:1775767509.002890:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767509.002907:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775767509.002908:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114717440:192.168.202.110@tcp:250:kworker.0
00000008:00100000:3.0:1775767509.002915:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767509.002930:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745880 x1862022114717696/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767509.002941:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767509.002945:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775767509.002947:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767509.002951:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4886
00000100:00080000:3.0:1775767509.002964:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767509.002968:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767509.002973:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:2.0:1775767509.002979:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767509.002988:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747480 x1862022114717824/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767509.002998:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767509.003002:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775767509.003003:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767509.003007:0:8:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4886
00000100:00080000:3.0:1775767509.003011:0:8:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767509.003015:0:8:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767509.003021:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775767509.003028:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767509/real 1775767509]  req@ffff9f1485747b80 x1862022114717440/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767534 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767509.003036:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744a80 x1862022114717952/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767509.003049:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767509.003052:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775767509.003053:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:2.0:1775767509.003058:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747b80 x1862022114717440/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767534 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767509.003060:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4886
00000100:00080000:3.0:1775767509.003066:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767509.003076:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00080000:2.0:1775767509.003079:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000008:00100000:3.0:1775767509.003083:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00080000:2.0:1775767509.003088:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767509.003094:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114717440:192.168.202.110@tcp:250:kworker.0
00000100:00100000:3.0:1775767509.003100:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746a00 x1862022114718080/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775767509.003120:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114717568:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775767509.003120:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767509.003129:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775767509.003132:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767509.003137:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4886
00000100:00080000:3.0:1775767509.003143:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767509.003146:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767509.003152:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000800:00000100:2.0:1775767509.003157:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767509.003166:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744380 x1862022114718208/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767509.003173:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:2.0:1775767509.003175:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767509/real 1775767509]  req@ffff9f1485746300 x1862022114717568/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767534 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767509.003178:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775767509.003181:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767509.003188:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4886
00000100:00100000:2.0:1775767509.003203:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746300 x1862022114717568/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767534 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767509.003204:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767509.003211:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767509.003217:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00080000:2.0:1775767509.003225:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00100000:3.0:1775767509.003229:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744e00 x1862022114718336/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767509.003235:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767509.003244:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114717568:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767509.003257:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114717696:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775767509.003282:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767509.003294:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767509/real 1775767509]  req@ffff9f1485745880 x1862022114717696/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767534 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767509.003308:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745880 x1862022114717696/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767534 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767509.003317:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767509.003320:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767509.003324:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114717696:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767509.003330:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114717824:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775767509.003352:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767509.003363:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767509/real 1775767509]  req@ffff9f1485747480 x1862022114717824/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767534 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767509.003375:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747480 x1862022114717824/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767534 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767509.003384:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767509.003387:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767509.003391:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114717824:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775767509.003397:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114717952:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775767509.003417:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767509.003428:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767509/real 1775767509]  req@ffff9f1485744a80 x1862022114717952/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767534 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767509.003441:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744a80 x1862022114717952/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767534 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767509.003449:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767509.003452:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767509.003456:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114717952:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775767509.003474:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114718080:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775767509.003493:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767509.003503:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767509/real 1775767509]  req@ffff9f1485746a00 x1862022114718080/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767534 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767509.003515:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746a00 x1862022114718080/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767534 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767509.003523:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767509.003527:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767509.003530:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114718080:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767509.003536:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114718208:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775767509.003552:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767509.003585:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767509/real 1775767509]  req@ffff9f1485744380 x1862022114718208/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767534 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767509.003597:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744380 x1862022114718208/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767534 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767509.003612:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767509.003615:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767509.003619:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114718208:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767509.003625:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114718336:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775767509.003645:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767509.003656:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767509/real 1775767509]  req@ffff9f1485744e00 x1862022114718336/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767534 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767509.003668:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744e00 x1862022114718336/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767534 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767509.003676:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767509.003679:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767509.003683:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114718336:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775767509.258711:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767510.026665:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767510.282743:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767511.050673:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775767511.306680:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767512.074634:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:0.0:1775767512.330726:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767513.098654:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775767513.354733:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1775767514.122686:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:2.0:1775767514.122694:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:1.0:1775767514.122696:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1775767514.122702:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767514.122714:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4891
00000100:00080000:1.0:1775767514.122735:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767514.122743:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775767514.122783:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ffb80 x1862022114718464/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767514.122812:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767514.122816:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775767514.122832:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767514.122838:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4891
00000100:00080000:1.0:1775767514.122842:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767514.122846:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767514.122873:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775767514.122890:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ffb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114718464:192.168.202.110@tcp:250:kworker.0
00000100:00100000:1.0:1775767514.122891:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fed80 x1862022114718592/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767514.122909:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767514.122912:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775767514.122913:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767514.122918:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4891
00000100:00080000:1.0:1775767514.122923:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767514.122927:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767514.122933:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767514.122946:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff800 x1862022114718720/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767514.122968:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767514.122972:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775767514.122977:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000800:00000100:2.0:1775767514.122977:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:1.0:1775767514.122986:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4891
00000100:00080000:1.0:1775767514.122991:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767514.123010:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767514.123016:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775767514.123029:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fca80 x1862022114718848/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767514.123039:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767514.123043:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1775767514.123045:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:2.0:1775767514.123051:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767514/real 1775767514]  req@ffff9f14ab4ffb80 x1862022114718464/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767539 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767514.123052:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4891
00000100:00080000:1.0:1775767514.123060:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767514.123068:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767514.123076:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775767514.123090:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ffb80 x1862022114718464/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767539 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767514.123093:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff100 x1862022114718976/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767514.123119:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767514.123126:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:1.0:1775767514.123133:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:2.0:1775767514.123134:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ffb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114718464:192.168.202.110@tcp:250:kworker.0
00000100:00080000:1.0:1775767514.123140:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775767514.123144:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767514.123149:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4891
00000100:00080000:1.0:1775767514.123154:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:2.0:1775767514.123163:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114718592:192.168.202.110@tcp:38:kworker.0
00000100:00080000:1.0:1775767514.123167:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767514.123176:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767514.123191:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdc00 x1862022114719104/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:2.0:1775767514.123199:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:1.0:1775767514.123219:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767514.123224:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775767514.123226:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:2.0:1775767514.123231:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767514/real 1775767514]  req@ffff9f14ab4fed80 x1862022114718592/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767539 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767514.123235:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4891
00000100:00080000:1.0:1775767514.123243:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767514.123250:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775767514.123255:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fed80 x1862022114718592/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767539 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:1.0:1775767514.123257:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00080000:2.0:1775767514.123282:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00100000:1.0:1775767514.123285:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fce00 x1862022114719232/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767514.123290:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767514.123298:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114718592:192.168.202.110@tcp:38:kworker.0
00000100:00080000:1.0:1775767514.123307:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767514.123313:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00100000:2.0:1775767514.123313:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114718720:192.168.202.110@tcp:38:kworker.0
00000100:00080000:1.0:1775767514.123316:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767514.123324:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4891
00000100:00080000:1.0:1775767514.123329:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767514.123345:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767514.123351:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:2.0:1775767514.123354:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767514.123368:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fea00 x1862022114719360/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775767514.123373:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767514/real 1775767514]  req@ffff9f14ab4ff800 x1862022114718720/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767539 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767514.123398:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff800 x1862022114718720/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767539 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767514.123424:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767514.123428:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767514.123432:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114718720:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767514.123438:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114718848:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775767514.123456:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767514.123466:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767514/real 1775767514]  req@ffff9f14ab4fca80 x1862022114718848/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767539 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767514.123477:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fca80 x1862022114718848/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767539 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767514.123484:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767514.123487:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767514.123490:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114718848:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775767514.123496:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114718976:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775767514.123514:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767514.123523:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767514/real 1775767514]  req@ffff9f14ab4ff100 x1862022114718976/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767539 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767514.123535:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff100 x1862022114718976/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767539 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767514.123542:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767514.123546:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767514.123549:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114718976:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775767514.123597:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114719104:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775767514.123617:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767514.123626:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767514/real 1775767514]  req@ffff9f14ab4fdc00 x1862022114719104/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767539 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767514.123637:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdc00 x1862022114719104/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767539 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767514.123655:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767514.123658:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767514.123661:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114719104:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767514.123667:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114719232:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775767514.123683:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767514.123692:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767514/real 1775767514]  req@ffff9f14ab4fce00 x1862022114719232/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767539 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767514.123704:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fce00 x1862022114719232/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767539 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767514.123712:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767514.123715:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767514.123718:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114719232:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767514.123736:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114719360:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775767514.123755:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767514.123767:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767514/real 1775767514]  req@ffff9f14ab4fea00 x1862022114719360/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767539 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767514.123779:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fea00 x1862022114719360/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767539 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767514.123787:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767514.123791:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767514.123794:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114719360:192.168.202.110@tcp:8:kworker.0
00010000:00010000:3.0:1775767514.378715:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767515.146670:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767515.402722:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767516.170658:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767516.426720:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767517.194744:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767517.450866:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767518.218666:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775767518.474682:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1775767519.242657:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767519.242664:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1775767519.242668:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767519.242677:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4896
00000100:00100000:0.0:1775767519.242686:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:1.0:1775767519.242687:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767519.242697:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775767519.242735:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fd880 x1862022114719488/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767519.242757:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767519.242761:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775767519.242762:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767519.242765:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4896
00000100:00080000:1.0:1775767519.242768:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767519.242793:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775767519.242801:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114719488:192.168.202.110@tcp:250:kworker.0
00000008:00100000:1.0:1775767519.242803:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767519.242821:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fc380 x1862022114719616/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767519.242829:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767519.242832:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775767519.242833:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767519.242837:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4896
00000100:00080000:1.0:1775767519.242854:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000800:00000100:0.0:1775767519.242863:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:1.0:1775767519.242865:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767519.242869:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767519.242881:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdf80 x1862022114719744/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767519.242889:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767519.242891:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775767519.242893:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767519.242897:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4896
00000100:00080000:1.0:1775767519.242900:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1775767519.242903:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767519/real 1775767519]  req@ffff9f14ab4fd880 x1862022114719488/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767544 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767519.242906:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767519.242913:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775767519.242932:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fd880 x1862022114719488/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767544 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767519.242941:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767519.242944:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767519.242948:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114719488:192.168.202.110@tcp:250:kworker.0
00000100:00100000:1.0:1775767519.242959:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fe300 x1862022114719872/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767519.242967:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767519.242970:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1775767519.242972:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775767519.242976:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114719616:192.168.202.110@tcp:38:kworker.0
00000100:00080000:1.0:1775767519.242979:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4896
00000100:00080000:1.0:1775767519.242986:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767519.242990:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767519.242994:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775767519.243005:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fd880 x1862022114720000/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1775767519.243014:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:1.0:1775767519.243014:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767519.243019:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775767519.243020:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767519.243025:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4896
00000100:00080000:1.0:1775767519.243028:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767519.243031:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775767519.243035:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767519/real 1775767519]  req@ffff9f14ab4fc380 x1862022114719616/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767544 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:1.0:1775767519.243036:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767519.243047:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fea00 x1862022114720128/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767519.243057:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1775767519.243060:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fc380 x1862022114719616/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767544 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767519.243064:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775767519.243069:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767519.243076:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767519.243079:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4896
00000100:00080000:0.0:1775767519.243082:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767519.243087:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114719616:192.168.202.110@tcp:38:kworker.0
00000100:00080000:1.0:1775767519.243087:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767519.243096:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775767519.243097:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114719744:192.168.202.110@tcp:38:kworker.0
00000008:00100000:1.0:1775767519.243108:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767519.243125:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fce00 x1862022114720256/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1775767519.243131:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:1.0:1775767519.243144:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767519.243148:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00100000:0.0:1775767519.243153:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767519/real 1775767519]  req@ffff9f14ab4fdf80 x1862022114719744/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767544 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767519.243153:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767519.243162:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4896
00000100:00080000:1.0:1775767519.243170:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767519.243176:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775767519.243179:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdf80 x1862022114719744/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767544 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:1.0:1775767519.243183:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00080000:0.0:1775767519.243200:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767519.243205:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767519.243211:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114719744:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775767519.243212:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdc00 x1862022114720384/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775767519.243225:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114719872:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767519.243253:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767519.243270:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767519/real 1775767519]  req@ffff9f14ab4fe300 x1862022114719872/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767544 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767519.243279:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fe300 x1862022114719872/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767544 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767519.243285:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767519.243288:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767519.243290:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114719872:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775767519.243301:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114720000:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767519.243313:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767519.243319:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767519/real 1775767519]  req@ffff9f14ab4fd880 x1862022114720000/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767544 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767519.243326:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fd880 x1862022114720000/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767544 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767519.243331:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767519.243334:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767519.243336:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114720000:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775767519.243340:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114720128:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767519.243350:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767519.243355:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767519/real 1775767519]  req@ffff9f14ab4fea00 x1862022114720128/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767544 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767519.243362:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fea00 x1862022114720128/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767544 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767519.243367:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767519.243368:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767519.243370:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114720128:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767519.243373:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114720256:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767519.243381:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767519.243386:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767519/real 1775767519]  req@ffff9f14ab4fce00 x1862022114720256/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767544 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767519.243393:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fce00 x1862022114720256/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767544 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767519.243398:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767519.243400:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767519.243402:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114720256:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767519.243405:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114720384:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767519.243413:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767519.243419:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767519/real 1775767519]  req@ffff9f14ab4fdc00 x1862022114720384/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767544 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767519.243427:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdc00 x1862022114720384/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767544 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767519.243432:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767519.243433:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767519.243435:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114720384:192.168.202.110@tcp:8:kworker.0
00010000:00010000:0.0:1775767519.498690:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767520.266714:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775767520.522744:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767521.290735:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767521.546827:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767522.314637:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767522.570763:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767523.338655:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767523.594790:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767524.362663:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775767524.362697:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767524.362705:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775767524.362710:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767524.362744:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4901
00000100:00080000:3.0:1775767524.362771:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767524.362777:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775767524.362812:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745500 x1862022114720512/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767524.362843:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767524.362846:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775767524.362848:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767524.362851:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4901
00000100:00080000:3.0:1775767524.362855:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767524.362858:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767524.362867:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767524.362877:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745f80 x1862022114720640/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767524.362886:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767524.362889:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775767524.362891:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767524.362894:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4901
00000100:00080000:3.0:1775767524.362897:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767524.362900:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767524.362905:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767524.362930:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747800 x1862022114720768/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775767524.362931:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114720512:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775767524.362949:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767524.362952:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775767524.362953:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767524.362956:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4901
00000100:00080000:3.0:1775767524.362959:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767524.362962:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767524.362965:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775767524.362973:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744000 x1862022114720896/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767524.362979:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767524.362982:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775767524.362983:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767524.362985:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4901
00000100:00080000:3.0:1775767524.362988:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767524.362991:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767524.362994:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775767524.363001:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746680 x1862022114721024/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1775767524.363005:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775767524.363014:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767524.363017:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775767524.363019:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767524.363023:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4901
00000100:00080000:3.0:1775767524.363025:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767524.363029:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767524.363032:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767524.363041:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745c00 x1862022114721152/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775767524.363045:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767524/real 1775767524]  req@ffff9f1485745500 x1862022114720512/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767549 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767524.363062:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767524.363066:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775767524.363067:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767524.363071:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4901
00000100:00100000:0.0:1775767524.363075:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745500 x1862022114720512/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767549 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767524.363075:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767524.363082:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767524.363089:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00080000:0.0:1775767524.363092:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767524.363098:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767524.363098:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745180 x1862022114721280/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775767524.363105:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114720512:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775767524.363114:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1775767524.363117:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114720640:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775767524.363119:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775767524.363123:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767524.363127:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4901
00000100:00080000:3.0:1775767524.363129:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767524.363132:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767524.363136:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:0.0:1775767524.363142:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767524.363146:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747100 x1862022114721408/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775767524.363159:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767524/real 1775767524]  req@ffff9f1485745f80 x1862022114720640/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767549 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767524.363176:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745f80 x1862022114720640/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767549 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767524.363186:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767524.363190:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767524.363205:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114720640:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767524.363225:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114720768:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767524.363242:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767524.363249:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767524/real 1775767524]  req@ffff9f1485747800 x1862022114720768/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767549 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767524.363258:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747800 x1862022114720768/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767549 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767524.363264:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767524.363267:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767524.363269:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114720768:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767524.363272:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114720896:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767524.363282:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767524.363288:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767524/real 1775767524]  req@ffff9f1485744000 x1862022114720896/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767549 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767524.363295:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744000 x1862022114720896/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767549 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767524.363302:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767524.363304:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767524.363306:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114720896:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775767524.363309:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114721024:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767524.363318:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767524.363324:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767524/real 1775767524]  req@ffff9f1485746680 x1862022114721024/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767549 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767524.363330:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746680 x1862022114721024/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767549 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767524.363338:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767524.363341:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767524.363344:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114721024:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775767524.363350:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114721152:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767524.363371:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767524.363382:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767524/real 1775767524]  req@ffff9f1485745c00 x1862022114721152/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767549 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767524.363395:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745c00 x1862022114721152/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767549 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767524.363404:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767524.363408:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767524.363412:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114721152:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767524.363416:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114721280:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767524.363432:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767524.363439:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767524/real 1775767524]  req@ffff9f1485745180 x1862022114721280/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767549 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767524.363447:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745180 x1862022114721280/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767549 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767524.363452:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767524.363455:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767524.363457:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114721280:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767524.363460:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114721408:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767524.363469:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767524.363475:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767524/real 1775767524]  req@ffff9f1485747100 x1862022114721408/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767549 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767524.363482:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747100 x1862022114721408/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767549 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767524.363487:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767524.363489:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767524.363491:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114721408:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775767524.618794:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767525.386657:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775767525.642739:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767526.410686:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:0.0:1775767526.666815:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767527.434645:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775767527.690747:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767528.458647:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767528.714735:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1775767529.482633:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1775767529.482640:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775767529.482645:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775767529.482651:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767529.482660:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4906
00000100:00080000:3.0:1775767529.482682:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767529.482689:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775767529.482719:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744700 x1862022114721536/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767529.482740:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767529.482743:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775767529.482745:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767529.482748:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4906
00000100:00080000:3.0:1775767529.482751:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767529.482754:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767529.482762:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767529.482772:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114721536:192.168.202.110@tcp:250:kworker.0
00000100:00100000:3.0:1775767529.482775:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747100 x1862022114721664/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767529.482786:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767529.482788:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775767529.482789:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767529.482793:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4906
00000100:00080000:3.0:1775767529.482795:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767529.482799:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767529.482802:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767529.482810:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745180 x1862022114721792/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767529.482816:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767529.482819:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775767529.482820:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767529.482822:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4906
00000100:00080000:3.0:1775767529.482824:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767529.482827:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767529.482830:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775767529.482836:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745c00 x1862022114721920/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767529.482841:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767529.482844:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775767529.482845:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767529.482848:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4906
00000100:00080000:3.0:1775767529.482851:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767529.482854:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767529.482857:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775767529.482863:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114721664:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767529.482868:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746680 x1862022114722048/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767529.482876:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767529.482879:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775767529.482880:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767529.482883:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4906
00000100:00080000:3.0:1775767529.482885:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767529.482889:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767529.482892:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767529.482901:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114721792:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767529.482903:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744000 x1862022114722176/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767529.482912:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767529.482915:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775767529.482917:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767529.482920:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4906
00000100:00100000:0.0:1775767529.482924:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114721920:192.168.202.110@tcp:8:kworker.0
00000100:00080000:3.0:1775767529.482925:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767529.482931:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767529.482935:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767529.482945:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747800 x1862022114722304/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775767529.482950:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114722048:192.168.202.110@tcp:8:kworker.0
00000100:00080000:3.0:1775767529.482958:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767529.482961:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775767529.482962:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767529.482966:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4906
00000100:00080000:3.0:1775767529.482970:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1775767529.482973:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114722176:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775767529.482975:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767529.482982:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775767529.482997:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114722304:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767529.483001:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745f80 x1862022114722432/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775767529.483021:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114722432:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775767529.738715:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767530.506744:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767530.762697:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767531.530627:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767531.786728:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767532.554657:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000400:00000100:1.0:1775767532.554833:0:2390:0:(lib-socket.c:454:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.202.110:988
00000400:00000100:1.0:1775767532.554876:0:2390:0:(acceptor.c:111:lnet_connect_console_error()) Connection to 192.168.202.110@tcp at host 192.168.202.110 was unreachable: the network or that node may be down, or Lustre may be misconfigured.
00000800:00000100:1.0:1775767532.554892:0:2390:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:1.0:1775767532.554933:0:2390:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:1.0:1775767532.554942:0:2390:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:1.0:1775767532.554951:0:2390:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:1.0:1775767532.554961:0:2390:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:1.0:1775767532.554971:0:2390:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:1.0:1775767532.554983:0:2390:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000100:00100000:0.0:1775767532.554995:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767529/real 1775767532]  req@ffff9f1485746680 x1862022114722048/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767554 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000800:00000100:1.0:1775767532.554996:0:2390:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000100:00100000:0.0:1775767532.555025:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746680 x1862022114722048/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767554 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767532.555037:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767532.555043:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767532.555048:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114722048:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775767532.555059:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767529/real 1775767532]  req@ffff9f1485744000 x1862022114722176/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767554 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767532.555081:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744000 x1862022114722176/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767554 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767532.555090:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767532.555094:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767532.555097:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114722176:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767532.555104:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767529/real 1775767532]  req@ffff9f1485747100 x1862022114721664/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767554 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767532.555116:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747100 x1862022114721664/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767554 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767532.555125:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767532.555129:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767532.555132:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114721664:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767532.555138:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767529/real 1775767532]  req@ffff9f1485745180 x1862022114721792/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767554 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767532.555150:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745180 x1862022114721792/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767554 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767532.555158:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767532.555161:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767532.555164:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114721792:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767532.555177:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767529/real 1775767532]  req@ffff9f1485745c00 x1862022114721920/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767554 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767532.555188:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745c00 x1862022114721920/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767554 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767532.555196:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767532.555199:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767532.555202:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114721920:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775767532.555209:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767529/real 1775767532]  req@ffff9f1485744700 x1862022114721536/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767554 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767532.555221:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744700 x1862022114721536/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767554 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767532.555228:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767532.555231:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767532.555234:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114721536:192.168.202.110@tcp:250:kworker.0
00000100:00100000:0.0:1775767532.555265:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767529/real 1775767532]  req@ffff9f1485745f80 x1862022114722432/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767554 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767532.555277:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745f80 x1862022114722432/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767554 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767532.555285:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767532.555289:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767532.555293:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114722432:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775767532.555301:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767529/real 1775767532]  req@ffff9f1485747800 x1862022114722304/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767554 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767532.555314:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747800 x1862022114722304/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767554 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767532.555322:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767532.555325:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767532.555328:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114722304:192.168.202.110@tcp:38:kworker.0
00010000:00010000:1.0:1775767532.810784:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767533.578673:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:0.0:1775767533.834692:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1775767534.602651:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:2.0:1775767534.602655:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775767534.602661:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775767534.602665:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767534.602674:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4912
00000100:00080000:3.0:1775767534.602684:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767534.602690:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775767534.602720:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745500 x1862022114722560/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767534.602748:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767534.602751:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775767534.602752:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767534.602756:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4912
00000100:00080000:3.0:1775767534.602758:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767534.602761:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767534.602768:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767534.602778:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744e00 x1862022114722688/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775767534.602785:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114722560:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775767534.602797:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767534.602801:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775767534.602803:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767534.602808:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4912
00000100:00080000:3.0:1775767534.602813:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767534.602818:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767534.602824:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767534.602837:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744380 x1862022114722816/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767534.602847:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767534.602851:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775767534.602853:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767534.602868:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4912
00000800:00000100:0.0:1775767534.602873:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775767534.602875:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767534.602881:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767534.602887:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775767534.602900:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746a00 x1862022114722944/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767534.602910:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767534.602914:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775767534.602916:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775767534.602917:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767534/real 1775767534]  req@ffff9f1485745500 x1862022114722560/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767559 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767534.602923:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4912
00000100:00080000:3.0:1775767534.602931:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767534.602936:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767534.602942:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775767534.602946:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745500 x1862022114722560/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767559 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767534.602959:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00100000:3.0:1775767534.602961:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744a80 x1862022114723072/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767534.602968:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767534.602979:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114722560:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775767534.603006:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767534.603009:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775767534.603011:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767534.603016:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4912
00000100:00080000:3.0:1775767534.603021:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1775767534.603022:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114722688:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775767534.603029:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767534.603035:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767534.603050:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747480 x1862022114723200/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1775767534.603058:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775767534.603067:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1775767534.603079:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767534/real 1775767534]  req@ffff9f1485744e00 x1862022114722688/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767559 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767534.603084:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775767534.603088:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767534.603098:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4912
00000100:00080000:3.0:1775767534.603105:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767534.603111:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775767534.603116:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744e00 x1862022114722688/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767559 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:3.0:1775767534.603118:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00080000:0.0:1775767534.603130:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00100000:3.0:1775767534.603135:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745880 x1862022114723328/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767534.603138:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767534.603147:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114722688:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775767534.603156:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1775767534.603161:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114722816:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775767534.603162:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775767534.603166:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767534.603173:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4912
00000100:00080000:3.0:1775767534.603178:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767534.603183:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000800:00000100:0.0:1775767534.603194:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000008:00100000:3.0:1775767534.603197:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775767534.603209:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767534/real 1775767534]  req@ffff9f1485744380 x1862022114722816/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767559 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767534.603214:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746300 x1862022114723456/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775767534.603243:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744380 x1862022114722816/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767559 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767534.603252:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767534.603257:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767534.603261:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114722816:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767534.603268:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114722944:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767534.603288:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767534.603299:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767534/real 1775767534]  req@ffff9f1485746a00 x1862022114722944/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767559 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767534.603311:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746a00 x1862022114722944/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767559 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767534.603323:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767534.603326:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767534.603329:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114722944:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775767534.603334:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114723072:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767534.603352:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767534.603363:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767534/real 1775767534]  req@ffff9f1485744a80 x1862022114723072/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767559 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767534.603374:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744a80 x1862022114723072/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767559 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767534.603381:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767534.603384:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767534.603387:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114723072:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775767534.603404:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114723200:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767534.603423:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767534.603434:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767534/real 1775767534]  req@ffff9f1485747480 x1862022114723200/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767559 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767534.603445:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747480 x1862022114723200/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767559 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767534.603453:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767534.603456:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767534.603461:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114723200:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767534.603467:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114723328:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767534.603489:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767534.603500:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767534/real 1775767534]  req@ffff9f1485745880 x1862022114723328/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767559 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767534.603512:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745880 x1862022114723328/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767559 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767534.603520:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767534.603523:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767534.603526:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114723328:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767534.603532:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114723456:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767534.603550:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767534.603598:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767534/real 1775767534]  req@ffff9f1485746300 x1862022114723456/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767559 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767534.603615:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746300 x1862022114723456/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767559 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767534.603634:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767534.603638:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767534.603640:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114723456:192.168.202.110@tcp:8:kworker.0
00010000:00010000:1.0:1775767534.858788:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767535.626652:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767535.882749:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767536.650681:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767536.906737:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767537.674693:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767537.930704:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767538.698685:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767538.954797:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767539.722648:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775767539.722655:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767539.722663:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775767539.722668:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767539.722677:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4917
00000100:00080000:3.0:1775767539.722687:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767539.722692:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775767539.722726:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747b80 x1862022114723584/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767539.722751:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767539.722754:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775767539.722756:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767539.722758:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4917
00000100:00080000:3.0:1775767539.722761:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767539.722763:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767539.722770:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767539.722779:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746300 x1862022114723712/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767539.722787:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767539.722790:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775767539.722792:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767539.722795:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4917
00000100:00080000:3.0:1775767539.722798:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767539.722800:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767539.722804:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767539.722811:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114723584:192.168.202.110@tcp:250:kworker.0
00000100:00100000:3.0:1775767539.722812:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745880 x1862022114723840/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767539.722825:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767539.722827:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775767539.722828:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767539.722831:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4917
00000100:00080000:3.0:1775767539.722833:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767539.722836:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767539.722838:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775767539.722847:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747480 x1862022114723968/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767539.722854:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767539.722856:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775767539.722857:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767539.722859:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4917
00000100:00080000:3.0:1775767539.722861:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767539.722864:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767539.722868:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:0.0:1775767539.722877:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767539.722877:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744a80 x1862022114724096/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767539.722887:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767539.722890:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775767539.722891:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767539.722895:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4917
00000100:00080000:3.0:1775767539.722898:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767539.722901:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767539.722903:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767539.722911:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746a00 x1862022114724224/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767539.722916:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1775767539.722918:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767539/real 1775767539]  req@ffff9f1485747b80 x1862022114723584/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767564 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767539.722919:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775767539.722923:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767539.722931:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4917
00000100:00080000:3.0:1775767539.722935:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767539.722937:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767539.722941:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767539.722943:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747b80 x1862022114723584/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767564 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767539.722951:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744380 x1862022114724352/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767539.722953:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767539.722962:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:3.0:1775767539.722964:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1775767539.722968:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114723584:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775767539.722969:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775767539.722974:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775767539.722980:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114723712:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775767539.722980:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4917
00000100:00080000:3.0:1775767539.722985:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767539.722989:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767539.722993:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775767539.723000:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744e00 x1862022114724480/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1775767539.723005:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767539.723020:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767539/real 1775767539]  req@ffff9f1485746300 x1862022114723712/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767564 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767539.723032:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746300 x1862022114723712/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767564 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767539.723038:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767539.723040:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767539.723041:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114723712:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767539.723058:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114723840:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767539.723068:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767539.723074:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767539/real 1775767539]  req@ffff9f1485745880 x1862022114723840/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767564 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767539.723082:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745880 x1862022114723840/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767564 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767539.723091:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767539.723094:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767539.723096:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114723840:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767539.723099:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114723968:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767539.723108:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767539.723113:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767539/real 1775767539]  req@ffff9f1485747480 x1862022114723968/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767564 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767539.723121:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747480 x1862022114723968/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767564 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767539.723126:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767539.723128:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767539.723129:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114723968:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775767539.723132:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114724096:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767539.723141:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767539.723147:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767539/real 1775767539]  req@ffff9f1485744a80 x1862022114724096/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767564 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767539.723154:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744a80 x1862022114724096/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767564 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767539.723159:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767539.723161:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767539.723163:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114724096:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775767539.723167:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114724224:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767539.723179:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767539.723187:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767539/real 1775767539]  req@ffff9f1485746a00 x1862022114724224/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767564 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767539.723199:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746a00 x1862022114724224/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767564 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767539.723207:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767539.723211:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767539.723214:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114724224:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767539.723220:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114724352:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767539.723238:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767539.723246:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767539/real 1775767539]  req@ffff9f1485744380 x1862022114724352/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767564 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767539.723256:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744380 x1862022114724352/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767564 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767539.723264:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767539.723268:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767539.723271:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114724352:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767539.723277:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114724480:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767539.723297:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767539.723309:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767539/real 1775767539]  req@ffff9f1485744e00 x1862022114724480/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767564 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767539.723321:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744e00 x1862022114724480/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767564 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767539.723326:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767539.723329:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767539.723331:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114724480:192.168.202.110@tcp:8:kworker.0
00010000:00010000:1.0:1775767539.978730:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767540.746677:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775767541.002695:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767541.770636:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775767542.026733:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767542.794645:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767543.050732:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767543.818673:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767544.074716:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1775767544.842678:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:2.0:1775767544.842686:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775767544.842690:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775767544.842697:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767544.842707:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4922
00000100:00080000:3.0:1775767544.842720:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767544.842728:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775767544.842774:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745500 x1862022114724608/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767544.842804:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767544.842808:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775767544.842809:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767544.842812:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4922
00000100:00080000:3.0:1775767544.842815:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767544.842817:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767544.842827:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767544.842836:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747800 x1862022114724736/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767544.842847:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767544.842851:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775767544.842853:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767544.842858:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4922
00000100:00080000:3.0:1775767544.842863:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767544.842867:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767544.842872:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767544.842881:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114724608:192.168.202.110@tcp:250:kworker.0
00000100:00100000:3.0:1775767544.842888:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745f80 x1862022114724864/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767544.842902:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767544.842906:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775767544.842908:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767544.842912:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4922
00000100:00080000:3.0:1775767544.842915:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767544.842918:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767544.842923:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775767544.842934:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744700 x1862022114724992/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767544.842941:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767544.842943:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775767544.842944:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767544.842947:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4922
00000100:00080000:3.0:1775767544.842951:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767544.842953:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767544.842957:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:0.0:1775767544.842966:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767544.842966:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745c00 x1862022114725120/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767544.842977:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767544.842981:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775767544.842984:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767544.842990:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4922
00000100:00080000:3.0:1775767544.842994:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767544.842999:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767544.843005:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767544.843018:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767544/real 1775767544]  req@ffff9f1485745500 x1862022114724608/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767569 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767544.843024:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745180 x1862022114725248/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767544.843048:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767544.843051:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775767544.843053:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767544.843056:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4922
00000100:00100000:0.0:1775767544.843061:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745500 x1862022114724608/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767569 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767544.843062:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767544.843068:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767544.843075:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00080000:0.0:1775767544.843088:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00100000:3.0:1775767544.843090:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747100 x1862022114725376/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767544.843097:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767544.843108:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114724608:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775767544.843110:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767544.843117:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775767544.843120:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767544.843124:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4922
00000100:00080000:3.0:1775767544.843127:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1775767544.843128:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114724736:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775767544.843132:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767544.843139:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775767544.843150:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744000 x1862022114725504/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1775767544.843172:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767544.843190:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767544/real 1775767544]  req@ffff9f1485747800 x1862022114724736/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767569 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767544.843206:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747800 x1862022114724736/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767569 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767544.843222:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767544.843226:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767544.843230:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114724736:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767544.843259:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114724864:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767544.843281:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767544.843292:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767544/real 1775767544]  req@ffff9f1485745f80 x1862022114724864/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767569 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767544.843305:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745f80 x1862022114724864/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767569 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767544.843313:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767544.843317:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767544.843321:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114724864:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767544.843328:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114724992:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767544.843351:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767544.843364:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767544/real 1775767544]  req@ffff9f1485744700 x1862022114724992/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767569 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767544.843377:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744700 x1862022114724992/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767569 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767544.843386:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767544.843389:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767544.843394:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114724992:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775767544.843401:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114725120:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767544.843420:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767544.843430:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767544/real 1775767544]  req@ffff9f1485745c00 x1862022114725120/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767569 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767544.843443:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745c00 x1862022114725120/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767569 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767544.843452:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767544.843455:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767544.843459:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114725120:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775767544.843465:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114725248:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767544.843481:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767544.843492:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767544/real 1775767544]  req@ffff9f1485745180 x1862022114725248/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767569 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767544.843504:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745180 x1862022114725248/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767569 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767544.843514:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767544.843517:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767544.843521:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114725248:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767544.843595:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114725376:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767544.843617:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767544.843628:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767544/real 1775767544]  req@ffff9f1485747100 x1862022114725376/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767569 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767544.843641:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747100 x1862022114725376/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767569 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767544.843651:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767544.843654:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767544.843658:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114725376:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767544.843664:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114725504:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767544.843681:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767544.843689:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767544/real 1775767544]  req@ffff9f1485744000 x1862022114725504/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767569 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767544.843697:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744000 x1862022114725504/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767569 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767544.843703:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767544.843705:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767544.843708:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114725504:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775767545.098776:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767545.866647:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767546.122719:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767546.890684:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775767547.146759:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767547.914647:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775767548.170706:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767548.938679:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775767549.194710:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767549.962710:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:0.0:1775767549.962722:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767549.962732:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1775767549.962738:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767549.962749:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4927
00000100:00080000:0.0:1775767549.962761:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767549.962768:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775767549.962804:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d1180 x1862022114725632/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767549.962836:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767549.962839:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775767549.962841:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767549.962846:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4927
00000100:00080000:0.0:1775767549.962850:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767549.962855:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767549.962865:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767549.962879:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3480 x1862022114725760/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775767549.962887:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114725632:192.168.202.110@tcp:250:kworker.0
00000100:00080000:0.0:1775767549.962899:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767549.962903:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:0.0:1775767549.962906:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767549.962910:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4927
00000100:00080000:0.0:1775767549.962915:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767549.962919:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767549.962925:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767549.962936:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0380 x1862022114725888/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767549.962952:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767549.962956:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775767549.962958:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767549.962963:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4927
00000100:00080000:0.0:1775767549.962968:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767549.962973:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767549.962980:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:2.0:1775767549.962993:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767549.962995:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d1f80 x1862022114726016/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767549.963007:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767549.963011:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1775767549.963014:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767549.963018:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4927
00000100:00080000:0.0:1775767549.963024:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767549.963029:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767549.963034:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775767549.963052:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3b80 x1862022114726144/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775767549.963063:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767549/real 1775767549]  req@ffff9f14a04d1180 x1862022114725632/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767574 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767549.963069:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767549.963083:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775767549.963087:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767549.963097:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4927
00000100:00080000:0.0:1775767549.963103:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:2.0:1775767549.963105:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1180 x1862022114725632/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767574 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767549.963113:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767549.963129:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00080000:2.0:1775767549.963131:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767549.963141:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767549.963151:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114725632:192.168.202.110@tcp:250:kworker.0
00000100:00100000:0.0:1775767549.963152:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3100 x1862022114726272/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767549.963174:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767549.963178:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:0.0:1775767549.963180:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767549.963185:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4927
00000100:00080000:0.0:1775767549.963189:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767549.963194:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767549.963200:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775767549.963200:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114725760:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767549.963226:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0e00 x1862022114726400/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767549.963239:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000800:00000100:2.0:1775767549.963248:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:0.0:1775767549.963250:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775767549.963254:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767549.963260:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4927
00000100:00080000:0.0:1775767549.963266:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:2.0:1775767549.963271:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767549/real 1775767549]  req@ffff9f14a04d3480 x1862022114725760/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767574 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767549.963272:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767549.963284:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775767549.963301:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3480 x1862022114725760/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767574 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767549.963306:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0a80 x1862022114726528/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767549.963347:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767549.963358:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767549.963365:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114725760:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767549.963374:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114725888:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775767549.963401:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767549.963414:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767549/real 1775767549]  req@ffff9f14a04d0380 x1862022114725888/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767574 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767549.963427:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0380 x1862022114725888/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767574 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767549.963435:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767549.963440:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767549.963444:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114725888:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767549.963451:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114726016:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775767549.963504:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767549.963518:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767549/real 1775767549]  req@ffff9f14a04d1f80 x1862022114726016/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767574 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767549.963530:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1f80 x1862022114726016/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767574 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767549.963540:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767549.963543:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767549.963547:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114726016:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775767549.963579:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114726144:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775767549.963601:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767549.963613:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767549/real 1775767549]  req@ffff9f14a04d3b80 x1862022114726144/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767574 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767549.963625:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3b80 x1862022114726144/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767574 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767549.963634:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767549.963638:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767549.963641:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114726144:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775767549.963650:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114726272:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775767549.963677:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767549.963692:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767549/real 1775767549]  req@ffff9f14a04d3100 x1862022114726272/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767574 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767549.963705:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3100 x1862022114726272/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767574 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767549.963713:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767549.963716:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767549.963722:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114726272:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767549.963745:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114726400:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775767549.963768:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767549.963779:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767549/real 1775767549]  req@ffff9f14a04d0e00 x1862022114726400/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767574 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767549.963791:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0e00 x1862022114726400/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767574 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767549.963799:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767549.963803:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767549.963806:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114726400:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767549.963813:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114726528:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775767549.963835:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767549.963846:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767549/real 1775767549]  req@ffff9f14a04d0a80 x1862022114726528/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767574 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767549.963857:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0a80 x1862022114726528/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767574 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767549.963865:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767549.963869:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767549.963872:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114726528:192.168.202.110@tcp:8:kworker.0
00010000:00010000:3.0:1775767550.218700:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767550.986677:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767551.242761:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767552.010718:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767552.266717:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767553.034702:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767553.290783:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767554.058692:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775767554.314746:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767555.082675:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775767555.082692:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767555.082702:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775767555.082707:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767555.082718:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4932
00000100:00080000:3.0:1775767555.082728:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767555.082736:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775767555.082777:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746680 x1862022114726656/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767555.082805:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767555.082807:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775767555.082809:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767555.082812:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4932
00000100:00080000:3.0:1775767555.082816:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767555.082820:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767555.082830:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767555.082842:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744000 x1862022114726784/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775767555.082852:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114726656:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775767555.082857:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767555.082866:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775767555.082867:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767555.082872:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4932
00000100:00080000:3.0:1775767555.082876:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767555.082879:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767555.082884:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767555.082907:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747100 x1862022114726912/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767555.082914:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767555.082916:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775767555.082918:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767555.082921:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4932
00000100:00080000:3.0:1775767555.082924:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767555.082928:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767555.082933:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775767555.082939:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745180 x1862022114727040/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767555.082945:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767555.082948:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775767555.082951:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767555.082954:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4932
00000100:00080000:3.0:1775767555.082958:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000800:00000100:0.0:1775767555.082969:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775767555.083013:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767555.083020:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775767555.083026:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767555/real 1775767555]  req@ffff9f1485746680 x1862022114726656/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767580 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767555.083038:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745c00 x1862022114727168/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767555.083072:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1775767555.083097:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746680 x1862022114726656/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767580 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767555.083107:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767555.083112:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767555.083116:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114726656:192.168.202.110@tcp:250:kworker.0
00000100:00100000:0.0:1775767555.083143:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114726784:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767555.083163:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767555.083173:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767555/real 1775767555]  req@ffff9f1485744000 x1862022114726784/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767580 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767555.083184:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744000 x1862022114726784/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767580 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767555.083192:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767555.083198:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767555.083203:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114726784:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767555.083211:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114726912:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767555.083226:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775767555.083232:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00100000:0.0:1775767555.083235:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767555/real 1775767555]  req@ffff9f1485747100 x1862022114726912/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767580 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767555.083237:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775767555.083247:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747100 x1862022114726912/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767580 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767555.083258:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767555.083261:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767555.083264:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114726912:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767555.083274:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114727040:192.168.202.110@tcp:8:kworker.0
00000100:00080000:3.0:1775767555.083296:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4932
00000800:00000100:0.0:1775767555.083303:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775767555.083313:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1775767555.083319:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767555/real 1775767555]  req@ffff9f1485745180 x1862022114727040/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767580 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767555.083325:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767555.083345:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767555.083354:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745180 x1862022114727040/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767580 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767555.083365:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00100000:3.0:1775767555.083371:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744700 x1862022114727296/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767555.083372:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767555.083380:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114727040:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775767555.083393:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114727168:192.168.202.110@tcp:8:kworker.0
00000100:00080000:3.0:1775767555.083415:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767555.083420:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000800:00000100:0.0:1775767555.083421:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775767555.083439:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767555.083449:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4932
00000100:00100000:0.0:1775767555.083455:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767555/real 1775767555]  req@ffff9f1485745c00 x1862022114727168/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767580 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767555.083455:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767555.083472:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775767555.083477:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745c00 x1862022114727168/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767580 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:3.0:1775767555.083483:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00080000:0.0:1775767555.083485:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767555.083496:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767555.083500:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114727168:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775767555.083511:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745f80 x1862022114727424/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775767555.083519:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114727296:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767555.083547:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775767555.083553:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1775767555.083596:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767555/real 1775767555]  req@ffff9f1485744700 x1862022114727296/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767580 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767555.083607:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744700 x1862022114727296/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767580 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767555.083614:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775767555.083623:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767555.083624:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767555.083638:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00080000:3.0:1775767555.083641:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4932
00000100:00100000:0.0:1775767555.083647:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114727296:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775767555.083650:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767555.083658:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775767555.083662:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114727424:192.168.202.110@tcp:38:kworker.0
00000008:00100000:3.0:1775767555.083669:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:0.0:1775767555.083687:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767555.083689:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747800 x1862022114727552/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775767555.083699:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767555/real 1775767555]  req@ffff9f1485745f80 x1862022114727424/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767580 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767555.083714:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745f80 x1862022114727424/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767580 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767555.083720:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767555.083726:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767555.083728:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114727424:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767555.083741:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114727552:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767555.083751:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767555.083758:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767555/real 1775767555]  req@ffff9f1485747800 x1862022114727552/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767580 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767555.083766:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747800 x1862022114727552/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767580 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767555.083771:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767555.083773:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767555.083775:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114727552:192.168.202.110@tcp:8:kworker.0
00010000:00010000:1.0:1775767555.338755:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767556.106686:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775767556.362779:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767557.130686:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767557.386705:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767558.154700:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767558.410794:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767559.178714:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767559.434746:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767560.202693:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:0.0:1775767560.202704:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767560.202712:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1775767560.202717:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767560.202731:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4937
00000100:00080000:0.0:1775767560.202745:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767560.202754:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775767560.202805:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d2300 x1862022114727680/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767560.202844:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767560.202847:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775767560.202849:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767560.202854:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4937
00000100:00080000:0.0:1775767560.202858:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767560.202863:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767560.202876:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775767560.202883:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114727680:192.168.202.110@tcp:250:kworker.0
00000100:00100000:0.0:1775767560.202894:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0000 x1862022114727808/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767560.202907:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767560.202911:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:0.0:1775767560.202913:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767560.202918:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4937
00000100:00080000:0.0:1775767560.202923:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767560.202926:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767560.202933:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767560.202945:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d1500 x1862022114727936/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767560.202954:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767560.202957:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775767560.202959:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767560.202965:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4937
00000800:00000100:2.0:1775767560.202966:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:0.0:1775767560.202972:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767560.202978:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767560.202984:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775767560.202997:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3800 x1862022114728064/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767560.203007:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767560.203010:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1775767560.203012:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:2.0:1775767560.203016:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767560/real 1775767560]  req@ffff9f14a04d2300 x1862022114727680/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767585 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767560.203021:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4937
00000100:00080000:0.0:1775767560.203029:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767560.203034:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775767560.203040:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d2300 x1862022114727680/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767585 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:0.0:1775767560.203041:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00080000:2.0:1775767560.203055:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00100000:0.0:1775767560.203060:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d2a00 x1862022114728192/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767560.203061:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767560.203068:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114727680:192.168.202.110@tcp:250:kworker.0
00000100:00080000:0.0:1775767560.203082:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767560.203085:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775767560.203087:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767560.203093:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4937
00000100:00100000:2.0:1775767560.203097:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114727808:192.168.202.110@tcp:38:kworker.0
00000100:00080000:0.0:1775767560.203098:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767560.203108:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767560.203113:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000800:00000100:2.0:1775767560.203121:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767560.203131:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d2300 x1862022114728320/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775767560.203134:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767560/real 1775767560]  req@ffff9f14a04d0000 x1862022114727808/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767585 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767560.203165:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767560.203171:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00100000:2.0:1775767560.203173:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0000 x1862022114727808/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767585 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767560.203174:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767560.203184:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4937
00000100:00080000:2.0:1775767560.203186:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767560.203192:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767560.203192:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767560.203198:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114727808:192.168.202.110@tcp:38:kworker.0
00000100:00080000:0.0:1775767560.203202:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775767560.203206:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114727936:192.168.202.110@tcp:38:kworker.0
00000008:00100000:0.0:1775767560.203211:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000800:00000100:2.0:1775767560.203222:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767560.203230:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0a80 x1862022114728448/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775767560.203232:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767560/real 1775767560]  req@ffff9f14a04d1500 x1862022114727936/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767585 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767560.203262:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1500 x1862022114727936/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767585 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767560.203280:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767560.203285:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767560.203285:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767560.203291:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114727936:192.168.202.110@tcp:38:kworker.0
00000100:00080000:0.0:1775767560.203293:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775767560.203297:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:2.0:1775767560.203300:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114728064:192.168.202.110@tcp:8:kworker.0
00000100:00080000:0.0:1775767560.203306:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4937
00000100:00080000:0.0:1775767560.203312:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767560.203316:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000800:00000100:2.0:1775767560.203320:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000008:00100000:0.0:1775767560.203325:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775767560.203329:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767560/real 1775767560]  req@ffff9f14a04d3800 x1862022114728064/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767585 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767560.203337:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3800 x1862022114728064/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767585 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767560.203344:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0e00 x1862022114728576/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767560.203391:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767560.203396:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767560.203401:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114728064:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775767560.203406:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114728192:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775767560.203424:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767560.203431:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767560/real 1775767560]  req@ffff9f14a04d2a00 x1862022114728192/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767585 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767560.203439:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d2a00 x1862022114728192/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767585 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767560.203445:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767560.203447:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767560.203449:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114728192:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775767560.203462:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114728320:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775767560.203472:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767560.203478:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767560/real 1775767560]  req@ffff9f14a04d2300 x1862022114728320/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767585 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767560.203485:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d2300 x1862022114728320/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767585 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767560.203491:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767560.203492:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767560.203495:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114728320:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767560.203499:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114728448:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775767560.203509:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767560.203514:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767560/real 1775767560]  req@ffff9f14a04d0a80 x1862022114728448/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767585 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767560.203521:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0a80 x1862022114728448/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767585 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767560.203526:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767560.203527:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767560.203529:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114728448:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767560.203532:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114728576:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775767560.203541:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767560.203546:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767560/real 1775767560]  req@ffff9f14a04d0e00 x1862022114728576/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767585 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767560.203552:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0e00 x1862022114728576/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767585 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767560.203578:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767560.203581:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767560.203583:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114728576:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775767560.458771:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767561.226653:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775767561.482721:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767562.250740:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775767562.506701:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767563.274654:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775767563.530732:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767564.298638:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767564.554732:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1775767565.322693:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767565.322700:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00100000:0.0:1775767565.322703:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775767565.322704:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767565.322724:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4942
00000100:00080000:3.0:1775767565.322746:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767565.322752:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775767565.322789:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745500 x1862022114728704/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767565.322816:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767565.322820:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775767565.322823:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767565.322828:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4942
00000100:00080000:3.0:1775767565.322833:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767565.322838:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767565.322847:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775767565.322852:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114728704:192.168.202.110@tcp:250:kworker.0
00000100:00100000:3.0:1775767565.322861:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744e00 x1862022114728832/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767565.322870:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767565.322872:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775767565.322874:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767565.322877:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4942
00000100:00080000:3.0:1775767565.322880:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767565.322882:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767565.322885:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767565.322893:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744380 x1862022114728960/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767565.322899:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767565.322902:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775767565.322903:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767565.322905:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4942
00000100:00080000:3.0:1775767565.322908:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000800:00000100:2.0:1775767565.322912:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775767565.322914:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767565.322918:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775767565.322927:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746a00 x1862022114729088/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767565.322932:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767565.322934:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775767565.322935:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767565.322939:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4942
00000100:00080000:3.0:1775767565.322941:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767565.322944:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767565.322948:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775767565.322952:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767565/real 1775767565]  req@ffff9f1485745500 x1862022114728704/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767590 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767565.322957:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744a80 x1862022114729216/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775767565.322973:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745500 x1862022114728704/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767590 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767565.322982:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767565.322989:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775767565.322990:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767565.322991:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767565.322995:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4942
00000100:00080000:2.0:1775767565.322999:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:3.0:1775767565.322999:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:2.0:1775767565.323006:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114728704:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775767565.323009:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767565.323016:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767565.323028:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747480 x1862022114729344/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767565.323038:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767565.323042:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775767565.323044:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:2.0:1775767565.323050:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114728832:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775767565.323051:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4942
00000100:00080000:3.0:1775767565.323058:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767565.323063:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767565.323067:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767565.323078:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745880 x1862022114729472/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:2.0:1775767565.323086:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775767565.323090:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767565.323094:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775767565.323095:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767565.323099:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4942
00000100:00080000:3.0:1775767565.323102:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767565.323106:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775767565.323107:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767565/real 1775767565]  req@ffff9f1485744e00 x1862022114728832/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767590 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:3.0:1775767565.323111:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775767565.323122:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746300 x1862022114729600/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775767565.323126:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744e00 x1862022114728832/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767590 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767565.323145:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767565.323148:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767565.323151:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114728832:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767565.323157:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114728960:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775767565.323173:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767565.323180:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767565/real 1775767565]  req@ffff9f1485744380 x1862022114728960/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767590 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767565.323201:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744380 x1862022114728960/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767590 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767565.323207:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767565.323209:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767565.323211:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114728960:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767565.323215:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114729088:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775767565.323226:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767565.323232:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767565/real 1775767565]  req@ffff9f1485746a00 x1862022114729088/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767590 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767565.323239:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746a00 x1862022114729088/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767590 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767565.323247:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767565.323250:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767565.323253:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114729088:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775767565.323259:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114729216:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775767565.323278:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767565.323289:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767565/real 1775767565]  req@ffff9f1485744a80 x1862022114729216/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767590 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767565.323302:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744a80 x1862022114729216/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767590 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767565.323313:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767565.323315:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767565.323317:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114729216:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775767565.323322:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114729344:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775767565.323354:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767565.323361:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767565/real 1775767565]  req@ffff9f1485747480 x1862022114729344/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767590 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767565.323369:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747480 x1862022114729344/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767590 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767565.323375:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767565.323377:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767565.323379:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114729344:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767565.323392:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114729472:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775767565.323403:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767565.323409:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767565/real 1775767565]  req@ffff9f1485745880 x1862022114729472/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767590 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767565.323417:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745880 x1862022114729472/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767590 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767565.323421:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767565.323423:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767565.323426:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114729472:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767565.323430:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114729600:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775767565.323439:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767565.323453:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767565/real 1775767565]  req@ffff9f1485746300 x1862022114729600/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767590 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767565.323460:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746300 x1862022114729600/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767590 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767565.323465:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767565.323467:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767565.323469:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114729600:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775767565.578704:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767566.346652:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767566.602841:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767567.370651:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767567.626711:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767568.394648:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775767568.650679:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767569.418646:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775767569.674690:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767570.442621:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775767570.442629:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767570.442635:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775767570.442638:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767570.442645:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4947
00000100:00080000:3.0:1775767570.442655:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767570.442678:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775767570.442711:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747b80 x1862022114729728/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767570.442736:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767570.442740:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775767570.442742:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767570.442746:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4947
00000100:00080000:3.0:1775767570.442750:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767570.442755:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767570.442766:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775767570.442784:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114729728:192.168.202.110@tcp:250:kworker.0
00000100:00100000:3.0:1775767570.442784:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746300 x1862022114729856/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767570.442799:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767570.442801:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775767570.442802:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767570.442806:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4947
00000100:00080000:3.0:1775767570.442809:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767570.442812:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767570.442816:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767570.442825:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745880 x1862022114729984/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767570.442831:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767570.442834:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775767570.442836:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767570.442838:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4947
00000100:00080000:3.0:1775767570.442840:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767570.442842:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767570.442846:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:2.0:1775767570.442847:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767570.442856:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747480 x1862022114730112/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767570.442863:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767570.442865:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775767570.442866:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767570.442869:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4947
00000100:00080000:3.0:1775767570.442871:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767570.442874:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767570.442877:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775767570.442886:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767570/real 1775767570]  req@ffff9f1485747b80 x1862022114729728/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767595 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767570.442887:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744a80 x1862022114730240/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767570.442916:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767570.442919:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775767570.442920:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:2.0:1775767570.442923:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747b80 x1862022114729728/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767595 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767570.442927:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4947
00000100:00080000:3.0:1775767570.442932:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767570.442938:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00080000:2.0:1775767570.442939:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000008:00100000:3.0:1775767570.442941:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00080000:2.0:1775767570.442946:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767570.442952:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114729728:192.168.202.110@tcp:250:kworker.0
00000100:00100000:3.0:1775767570.442952:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746a00 x1862022114730368/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767570.442971:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767570.442974:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775767570.442977:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:2.0:1775767570.442979:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114729856:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775767570.442986:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4947
00000100:00080000:3.0:1775767570.442991:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767570.442996:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000800:00000100:2.0:1775767570.443001:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000008:00100000:3.0:1775767570.443004:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775767570.443012:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767570/real 1775767570]  req@ffff9f1485746300 x1862022114729856/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767595 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767570.443020:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744380 x1862022114730496/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775767570.443023:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746300 x1862022114729856/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767595 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767570.443048:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767570.443048:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767570.443054:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775767570.443056:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00080000:3.0:1775767570.443057:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:2.0:1775767570.443062:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114729856:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775767570.443063:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4947
00000100:00080000:3.0:1775767570.443071:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:2.0:1775767570.443074:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114729984:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775767570.443077:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767570.443083:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:2.0:1775767570.443093:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767570.443096:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744e00 x1862022114730624/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775767570.443103:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767570/real 1775767570]  req@ffff9f1485745880 x1862022114729984/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767595 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767570.443118:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745880 x1862022114729984/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767595 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767570.443127:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767570.443131:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767570.443135:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114729984:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767570.443141:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114730112:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775767570.443164:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767570.443176:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767570/real 1775767570]  req@ffff9f1485747480 x1862022114730112/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767595 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767570.443189:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747480 x1862022114730112/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767595 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767570.443195:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767570.443197:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767570.443199:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114730112:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775767570.443203:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114730240:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775767570.443218:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767570.443227:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767570/real 1775767570]  req@ffff9f1485744a80 x1862022114730240/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767595 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767570.443238:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744a80 x1862022114730240/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767595 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767570.443246:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767570.443249:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767570.443252:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114730240:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775767570.443259:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114730368:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775767570.443295:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767570.443306:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767570/real 1775767570]  req@ffff9f1485746a00 x1862022114730368/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767595 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767570.443314:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746a00 x1862022114730368/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767595 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767570.443324:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767570.443326:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767570.443328:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114730368:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767570.443341:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114730496:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775767570.443354:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767570.443361:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767570/real 1775767570]  req@ffff9f1485744380 x1862022114730496/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767595 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767570.443368:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744380 x1862022114730496/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767595 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767570.443373:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767570.443374:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767570.443376:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114730496:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767570.443380:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114730624:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775767570.443389:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767570.443394:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767570/real 1775767570]  req@ffff9f1485744e00 x1862022114730624/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767595 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767570.443402:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744e00 x1862022114730624/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767595 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767570.443406:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767570.443408:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767570.443411:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114730624:192.168.202.110@tcp:8:kworker.0
00010000:00010000:1.0:1775767570.698699:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767571.466637:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767571.722710:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767572.490655:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767572.746719:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767573.514643:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767573.770725:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767574.538648:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767574.794754:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767575.562677:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:0.0:1775767575.562783:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767575.562790:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1775767575.562794:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767575.562804:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4953
00000100:00080000:0.0:1775767575.562815:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767575.562822:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775767575.562859:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3100 x1862022114730752/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767575.562886:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767575.562890:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775767575.562892:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767575.562896:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4953
00000100:00080000:0.0:1775767575.562901:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767575.562905:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767575.562916:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775767575.562927:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114730752:192.168.202.110@tcp:250:kworker.0
00000100:00100000:0.0:1775767575.562933:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3b80 x1862022114730880/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767575.562943:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767575.562946:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:0.0:1775767575.562947:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767575.562950:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4953
00000100:00080000:0.0:1775767575.562953:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767575.562956:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767575.562960:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767575.562969:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d1f80 x1862022114731008/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767575.562979:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767575.562982:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775767575.562983:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767575.562987:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4953
00000100:00080000:0.0:1775767575.562989:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767575.562991:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000800:00000100:2.0:1775767575.562993:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000008:00100000:0.0:1775767575.562995:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775767575.563003:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0380 x1862022114731136/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767575.563013:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767575.563016:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1775767575.563018:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767575.563022:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4953
00000100:00080000:0.0:1775767575.563027:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00100000:2.0:1775767575.563031:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767575/real 1775767575]  req@ffff9f14a04d3100 x1862022114730752/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767600 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767575.563032:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767575.563041:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775767575.563051:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3100 x1862022114730752/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767600 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767575.563056:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3480 x1862022114731264/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767575.563062:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767575.563071:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:0.0:1775767575.563074:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:2.0:1775767575.563077:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114730752:192.168.202.110@tcp:250:kworker.0
00000100:00080000:0.0:1775767575.563079:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775767575.563082:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767575.563087:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4953
00000100:00080000:0.0:1775767575.563091:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:2.0:1775767575.563099:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114730880:192.168.202.110@tcp:38:kworker.0
00000100:00080000:0.0:1775767575.563113:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000800:00000100:2.0:1775767575.563118:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000008:00100000:0.0:1775767575.563120:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775767575.563128:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767575/real 1775767575]  req@ffff9f14a04d3b80 x1862022114730880/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767600 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767575.563137:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d1180 x1862022114731392/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775767575.563139:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3b80 x1862022114730880/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767600 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767575.563166:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767575.563168:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767575.563172:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00080000:0.0:1775767575.563174:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00100000:2.0:1775767575.563177:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114730880:192.168.202.110@tcp:38:kworker.0
00000100:00080000:0.0:1775767575.563178:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767575.563187:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4953
00000100:00080000:0.0:1775767575.563192:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:2.0:1775767575.563194:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114731008:192.168.202.110@tcp:38:kworker.0
00000100:00080000:0.0:1775767575.563198:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767575.563204:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000800:00000100:2.0:1775767575.563213:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767575.563217:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3100 x1862022114731520/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767575.563242:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767575.563245:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775767575.563246:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767575.563250:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4953
00000100:00100000:2.0:1775767575.563251:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767575/real 1775767575]  req@ffff9f14a04d1f80 x1862022114731008/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767600 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767575.563253:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767575.563260:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767575.563268:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775767575.563275:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1f80 x1862022114731008/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767600 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767575.563279:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0e00 x1862022114731648/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767575.563295:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767575.563304:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767575.563311:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114731008:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767575.563319:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114731136:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775767575.563336:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767575.563343:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767575/real 1775767575]  req@ffff9f14a04d0380 x1862022114731136/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767600 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767575.563351:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0380 x1862022114731136/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767600 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767575.563356:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767575.563358:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767575.563360:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114731136:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775767575.563363:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114731264:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775767575.563373:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767575.563378:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767575/real 1775767575]  req@ffff9f14a04d3480 x1862022114731264/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767600 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767575.563385:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3480 x1862022114731264/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767600 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767575.563390:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767575.563392:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767575.563394:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114731264:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775767575.563411:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114731392:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775767575.563430:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767575.563442:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767575/real 1775767575]  req@ffff9f14a04d1180 x1862022114731392/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767600 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767575.563454:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1180 x1862022114731392/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767600 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767575.563463:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767575.563466:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767575.563470:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114731392:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767575.563476:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114731520:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775767575.563496:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767575.563507:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767575/real 1775767575]  req@ffff9f14a04d3100 x1862022114731520/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767600 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767575.563519:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3100 x1862022114731520/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767600 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767575.563534:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767575.563538:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767575.563541:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114731520:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767575.563547:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114731648:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775767575.563595:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767575.563605:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767575/real 1775767575]  req@ffff9f14a04d0e00 x1862022114731648/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767600 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767575.563617:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0e00 x1862022114731648/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767600 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767575.563623:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767575.563625:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767575.563628:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114731648:192.168.202.110@tcp:8:kworker.0
00010000:00010000:1.0:1775767575.818737:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767576.586702:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767576.842713:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767577.610702:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775767577.866718:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767578.634640:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767578.890690:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767579.658701:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767579.914698:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767580.682636:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775767580.682657:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767580.682665:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775767580.682668:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767580.682676:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4958
00000100:00080000:3.0:1775767580.682685:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767580.682690:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775767580.682736:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745500 x1862022114731776/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767580.682756:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767580.682758:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775767580.682759:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767580.682762:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4958
00000100:00080000:3.0:1775767580.682765:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767580.682768:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767580.682776:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767580.682787:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747800 x1862022114731904/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775767580.682788:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114731776:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775767580.682804:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767580.682808:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775767580.682810:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767580.682816:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4958
00000100:00080000:3.0:1775767580.682820:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767580.682824:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767580.682830:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767580.682843:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745f80 x1862022114732032/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:2.0:1775767580.682851:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775767580.682860:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767580.682862:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775767580.682864:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767580.682869:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4958
00000100:00080000:3.0:1775767580.682873:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767580.682876:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767580.682881:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775767580.682891:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744700 x1862022114732160/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767580.682898:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:2.0:1775767580.682900:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767580/real 1775767580]  req@ffff9f1485745500 x1862022114731776/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767605 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767580.682901:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775767580.682904:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767580.682909:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4958
00000100:00080000:3.0:1775767580.682914:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767580.682918:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767580.682923:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775767580.682935:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745c00 x1862022114732288/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767580.682942:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:2.0:1775767580.682944:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745500 x1862022114731776/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767605 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767580.682945:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775767580.682948:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767580.682955:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4958
00000100:00080000:3.0:1775767580.682961:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767580.682963:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767580.682965:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767580.682970:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00080000:2.0:1775767580.682971:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767580.682976:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114731776:192.168.202.110@tcp:250:kworker.0
00000100:00100000:3.0:1775767580.682980:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745180 x1862022114732416/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767580.682987:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767580.682989:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775767580.682991:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767580.682993:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4958
00000100:00080000:3.0:1775767580.682996:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:2.0:1775767580.683000:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114731904:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775767580.683000:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767580.683007:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767580.683015:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747100 x1862022114732544/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767580.683022:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767580.683024:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775767580.683025:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767580.683028:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4958
00000100:00080000:3.0:1775767580.683032:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767580.683035:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000800:00000100:2.0:1775767580.683039:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000008:00100000:3.0:1775767580.683041:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775767580.683052:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744000 x1862022114732672/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775767580.683054:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767580/real 1775767580]  req@ffff9f1485747800 x1862022114731904/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767605 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767580.683081:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747800 x1862022114731904/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767605 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767580.683087:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767580.683090:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767580.683093:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114731904:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767580.683097:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114732032:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775767580.683109:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767580.683115:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767580/real 1775767580]  req@ffff9f1485745f80 x1862022114732032/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767605 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767580.683123:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745f80 x1862022114732032/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767605 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767580.683132:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767580.683134:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767580.683136:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114732032:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767580.683144:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114732160:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775767580.683155:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767580.683160:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767580/real 1775767580]  req@ffff9f1485744700 x1862022114732160/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767605 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767580.683167:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744700 x1862022114732160/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767605 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767580.683172:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767580.683174:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767580.683189:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114732160:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775767580.683193:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114732288:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775767580.683207:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767580.683214:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767580/real 1775767580]  req@ffff9f1485745c00 x1862022114732288/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767605 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767580.683225:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745c00 x1862022114732288/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767605 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767580.683233:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767580.683236:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767580.683240:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114732288:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775767580.683245:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114732416:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775767580.683266:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767580.683276:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767580/real 1775767580]  req@ffff9f1485745180 x1862022114732416/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767605 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767580.683288:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745180 x1862022114732416/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767605 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767580.683294:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767580.683296:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767580.683298:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114732416:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767580.683311:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114732544:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775767580.683325:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767580.683332:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767580/real 1775767580]  req@ffff9f1485747100 x1862022114732544/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767605 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767580.683340:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747100 x1862022114732544/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767605 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767580.683345:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767580.683347:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767580.683348:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114732544:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767580.683352:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114732672:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775767580.683361:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767580.683367:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767580/real 1775767580]  req@ffff9f1485744000 x1862022114732672/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767605 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767580.683374:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744000 x1862022114732672/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767605 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767580.683379:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767580.683381:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767580.683391:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114732672:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775767580.938716:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767581.706655:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767581.962748:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767582.730615:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775767582.986717:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767583.754690:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767584.010715:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767584.778640:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775767585.034761:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767585.802667:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:1.0:1775767585.802679:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767585.802688:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1775767585.802692:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767585.802704:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4963
00000100:00080000:1.0:1775767585.802715:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767585.802722:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775767585.802758:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff100 x1862022114732800/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767585.802786:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767585.802791:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775767585.802793:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767585.802797:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4963
00000100:00080000:1.0:1775767585.802803:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767585.802807:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767585.802817:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775767585.802830:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114732800:192.168.202.110@tcp:250:kworker.0
00000100:00100000:1.0:1775767585.802832:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fca80 x1862022114732928/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767585.802850:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767585.802854:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775767585.802856:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767585.802862:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4963
00000100:00080000:1.0:1775767585.802866:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767585.802870:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767585.802875:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767585.802889:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff800 x1862022114733056/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767585.802901:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767585.802905:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775767585.802907:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767585.802911:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4963
00000100:00080000:1.0:1775767585.802916:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000800:00000100:2.0:1775767585.802918:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:1.0:1775767585.802926:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767585.802934:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775767585.802949:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fed80 x1862022114733184/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767585.802960:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767585.802963:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1775767585.802965:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:2.0:1775767585.802972:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767585/real 1775767585]  req@ffff9f14ab4ff100 x1862022114732800/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767610 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767585.802973:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4963
00000100:00080000:1.0:1775767585.802987:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767585.802997:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775767585.803003:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff100 x1862022114732800/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767610 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:1.0:1775767585.803004:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00080000:2.0:1775767585.803023:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00100000:1.0:1775767585.803028:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ffb80 x1862022114733312/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767585.803032:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767585.803045:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114732800:192.168.202.110@tcp:250:kworker.0
00000100:00080000:1.0:1775767585.803075:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767585.803080:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775767585.803083:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767585.803089:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4963
00000100:00080000:1.0:1775767585.803094:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:2.0:1775767585.803095:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114732928:192.168.202.110@tcp:38:kworker.0
00000100:00080000:1.0:1775767585.803105:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767585.803112:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767585.803147:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff100 x1862022114733440/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:2.0:1775767585.803155:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:1.0:1775767585.803170:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:2.0:1775767585.803174:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767585/real 1775767585]  req@ffff9f14ab4fca80 x1862022114732928/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767610 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767585.803181:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775767585.803190:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:2.0:1775767585.803197:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fca80 x1862022114732928/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767610 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767585.803198:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4963
00000100:00080000:1.0:1775767585.803209:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767585.803212:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767585.803218:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00080000:1.0:1775767585.803221:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775767585.803225:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114732928:192.168.202.110@tcp:38:kworker.0
00000008:00100000:1.0:1775767585.803233:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775767585.803238:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114733056:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775767585.803259:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767585.803263:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdc00 x1862022114733568/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775767585.803270:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767585/real 1775767585]  req@ffff9f14ab4ff800 x1862022114733056/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767610 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767585.803320:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff800 x1862022114733056/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767610 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767585.803325:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767585.803331:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767585.803335:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775767585.803340:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767585.803340:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767585.803350:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114733056:192.168.202.110@tcp:38:kworker.0
00000100:00080000:1.0:1775767585.803351:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4963
00000100:00080000:1.0:1775767585.803360:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767585.803369:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775767585.803369:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114733184:192.168.202.110@tcp:8:kworker.0
00000008:00100000:1.0:1775767585.803383:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775767585.803400:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fce00 x1862022114733696/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:2.0:1775767585.803409:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767585.803426:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767585/real 1775767585]  req@ffff9f14ab4fed80 x1862022114733184/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767610 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767585.803437:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fed80 x1862022114733184/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767610 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767585.803446:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767585.803451:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767585.803455:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114733184:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775767585.803462:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ffb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114733312:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775767585.803483:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767585.803494:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767585/real 1775767585]  req@ffff9f14ab4ffb80 x1862022114733312/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767610 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767585.803514:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ffb80 x1862022114733312/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767610 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767585.803519:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767585.803522:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767585.803524:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ffb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114733312:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775767585.803540:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114733440:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775767585.803587:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767585.803601:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767585/real 1775767585]  req@ffff9f14ab4ff100 x1862022114733440/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767610 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767585.803614:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff100 x1862022114733440/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767610 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767585.803628:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767585.803630:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767585.803633:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114733440:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767585.803637:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114733568:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775767585.803654:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767585.803661:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767585/real 1775767585]  req@ffff9f14ab4fdc00 x1862022114733568/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767610 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767585.803669:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdc00 x1862022114733568/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767610 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767585.803674:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767585.803677:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767585.803679:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114733568:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767585.803682:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114733696:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775767585.803692:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767585.803697:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767585/real 1775767585]  req@ffff9f14ab4fce00 x1862022114733696/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767610 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767585.803704:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fce00 x1862022114733696/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767610 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767585.803709:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767585.803711:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767585.803714:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114733696:192.168.202.110@tcp:8:kworker.0
00010000:00010000:3.0:1775767586.058669:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767586.826661:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767587.082723:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767587.850659:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767588.106823:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767588.874701:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767589.130731:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767589.898660:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775767590.154745:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767590.922668:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:1.0:1775767590.922670:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767590.922681:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1775767590.922686:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767590.922699:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4968
00000100:00080000:1.0:1775767590.922711:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767590.922719:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775767590.922761:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fea00 x1862022114733824/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767590.922788:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767590.922790:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775767590.922792:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767590.922795:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4968
00000100:00080000:1.0:1775767590.922798:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767590.922801:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767590.922810:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767590.922822:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fd880 x1862022114733952/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775767590.922827:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114733824:192.168.202.110@tcp:250:kworker.0
00000100:00080000:1.0:1775767590.922848:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767590.922850:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775767590.922851:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767590.922855:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4968
00000100:00080000:1.0:1775767590.922857:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767590.922860:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767590.922863:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767590.922871:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fe300 x1862022114734080/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767590.922878:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767590.922881:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775767590.922883:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767590.922886:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4968
00000100:00080000:1.0:1775767590.922892:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767590.922895:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767590.922898:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775767590.922907:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdf80 x1862022114734208/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767590.922913:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767590.922916:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1775767590.922917:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767590.922920:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4968
00000800:00000100:2.0:1775767590.922928:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:1.0:1775767590.922930:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767590.922936:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767590.922940:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775767590.922950:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fc380 x1862022114734336/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767590.922957:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767590.922960:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775767590.922962:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767590.922966:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4968
00000100:00080000:1.0:1775767590.922969:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767590.922971:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767590.922975:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775767590.922982:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767590/real 1775767590]  req@ffff9f14ab4fea00 x1862022114733824/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767615 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767590.922988:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fce00 x1862022114734464/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767590.923028:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767590.923031:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775767590.923033:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767590.923037:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4968
00000100:00100000:2.0:1775767590.923040:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fea00 x1862022114733824/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767615 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767590.923041:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767590.923047:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767590.923056:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767590.923070:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdc00 x1862022114734592/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767590.923072:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767590.923098:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767590.923102:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775767590.923104:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:1.0:1775767590.923107:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:2.0:1775767590.923111:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114733824:192.168.202.110@tcp:250:kworker.0
00000100:00080000:1.0:1775767590.923119:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4968
00000100:00080000:1.0:1775767590.923124:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767590.923129:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767590.923134:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775767590.923146:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff100 x1862022114734720/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775767590.923150:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114733952:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775767590.923191:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767590.923206:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767590/real 1775767590]  req@ffff9f14ab4fd880 x1862022114733952/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767615 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767590.923219:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fd880 x1862022114733952/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767615 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767590.923237:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767590.923241:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767590.923245:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114733952:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767590.923256:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114734080:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775767590.923288:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767590.923303:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767590/real 1775767590]  req@ffff9f14ab4fe300 x1862022114734080/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767615 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767590.923317:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fe300 x1862022114734080/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767615 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767590.923326:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767590.923332:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767590.923337:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114734080:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767590.923348:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114734208:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775767590.923380:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767590.923393:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767590/real 1775767590]  req@ffff9f14ab4fdf80 x1862022114734208/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767615 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767590.923410:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdf80 x1862022114734208/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767615 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767590.923416:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767590.923419:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767590.923423:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114734208:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775767590.923430:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114734336:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775767590.923446:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767590.923456:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767590/real 1775767590]  req@ffff9f14ab4fc380 x1862022114734336/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767615 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767590.923467:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fc380 x1862022114734336/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767615 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767590.923475:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767590.923477:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767590.923480:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114734336:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775767590.923486:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114734464:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775767590.923501:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767590.923509:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767590/real 1775767590]  req@ffff9f14ab4fce00 x1862022114734464/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767615 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767590.923518:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fce00 x1862022114734464/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767615 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767590.923526:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767590.923532:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767590.923535:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114734464:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767590.923546:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114734592:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775767590.923588:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767590.923605:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767590/real 1775767590]  req@ffff9f14ab4fdc00 x1862022114734592/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767615 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767590.923615:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdc00 x1862022114734592/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767615 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767590.923621:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767590.923624:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767590.923627:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114734592:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767590.923651:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114734720:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775767590.923667:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767590.923678:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767590/real 1775767590]  req@ffff9f14ab4ff100 x1862022114734720/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767615 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767590.923689:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff100 x1862022114734720/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767615 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767590.923694:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767590.923697:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767590.923700:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114734720:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775767591.178720:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767591.946691:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775767592.202671:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767592.970707:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767593.226703:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767593.994696:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767594.250740:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767595.018671:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767595.274742:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1775767596.042665:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:2.0:1775767596.042666:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775767596.042676:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775767596.042681:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767596.042691:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4973
00000100:00080000:3.0:1775767596.042703:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767596.042709:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775767596.042741:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746680 x1862022114734848/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767596.042763:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767596.042765:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775767596.042767:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767596.042770:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4973
00000100:00080000:3.0:1775767596.042772:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767596.042774:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767596.042782:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767596.042792:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744000 x1862022114734976/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767596.042800:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:2.0:1775767596.042804:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114734848:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775767596.042811:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775767596.042813:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767596.042817:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4973
00000100:00080000:3.0:1775767596.042819:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767596.042821:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767596.042825:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767596.042833:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747100 x1862022114735104/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767596.042843:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767596.042846:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775767596.042847:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767596.042849:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4973
00000100:00080000:3.0:1775767596.042858:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767596.042860:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767596.042863:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775767596.042870:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745180 x1862022114735232/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767596.042876:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767596.042879:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775767596.042880:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767596.042882:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4973
00000100:00080000:3.0:1775767596.042884:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767596.042886:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767596.042889:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775767596.042900:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745c00 x1862022114735360/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767596.042906:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767596.042909:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775767596.042910:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767596.042913:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4973
00000100:00080000:3.0:1775767596.042917:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767596.042920:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775767596.042924:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114734976:192.168.202.110@tcp:38:kworker.0
00000008:00100000:3.0:1775767596.042926:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767596.042938:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744700 x1862022114735488/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767596.042945:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767596.042948:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775767596.042950:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767596.042953:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4973
00000100:00080000:3.0:1775767596.042956:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767596.042958:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767596.042961:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775767596.042969:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114735104:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767596.042973:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745f80 x1862022114735616/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767596.042985:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767596.042988:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775767596.042989:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767596.042991:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4973
00000100:00080000:3.0:1775767596.042994:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767596.042997:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767596.043001:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775767596.043010:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747800 x1862022114735744/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775767596.043018:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114735232:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775767596.043072:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114735360:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775767596.043099:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114735488:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767596.043119:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114735616:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767596.043130:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114735744:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775767596.298695:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767597.066644:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767597.322721:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767598.090636:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767598.346722:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767599.114668:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000400:00000100:0.0:1775767599.114801:0:2392:0:(lib-socket.c:454:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.202.110:988
00000400:00000100:0.0:1775767599.114834:0:2392:0:(acceptor.c:111:lnet_connect_console_error()) Connection to 192.168.202.110@tcp at host 192.168.202.110 was unreachable: the network or that node may be down, or Lustre may be misconfigured.
00000800:00000100:0.0:1775767599.114846:0:2392:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:0.0:1775767599.114877:0:2392:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:0.0:1775767599.114883:0:2392:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:0.0:1775767599.114889:0:2392:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:0.0:1775767599.114897:0:2392:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:0.0:1775767599.114902:0:2392:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:0.0:1775767599.114910:0:2392:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000100:00100000:2.0:1775767599.114922:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767596/real 1775767599]  req@ffff9f1485744700 x1862022114735488/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767621 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000800:00000100:0.0:1775767599.114923:0:2392:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000100:00100000:2.0:1775767599.114959:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744700 x1862022114735488/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767621 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767599.114972:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767599.114977:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767599.114982:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114735488:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767599.114993:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767596/real 1775767599]  req@ffff9f1485745f80 x1862022114735616/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767621 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767599.115030:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745f80 x1862022114735616/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767621 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767599.115040:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767599.115044:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767599.115047:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114735616:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767599.115055:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767596/real 1775767599]  req@ffff9f1485747800 x1862022114735744/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767621 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767599.115068:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747800 x1862022114735744/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767621 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767599.115077:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767599.115081:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767599.115094:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114735744:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775767599.115102:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767596/real 1775767599]  req@ffff9f1485744000 x1862022114734976/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767621 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767599.115115:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744000 x1862022114734976/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767621 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767599.115123:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767599.115127:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767599.115131:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114734976:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767599.115139:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767596/real 1775767599]  req@ffff9f1485747100 x1862022114735104/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767621 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767599.115152:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747100 x1862022114735104/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767621 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767599.115159:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767599.115163:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767599.115167:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114735104:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767599.115180:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767596/real 1775767599]  req@ffff9f1485745180 x1862022114735232/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767621 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767599.115194:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745180 x1862022114735232/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767621 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767599.115202:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767599.115206:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767599.115209:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114735232:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775767599.115216:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767596/real 1775767599]  req@ffff9f1485745c00 x1862022114735360/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767621 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767599.115228:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745c00 x1862022114735360/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767621 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767599.115237:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767599.115240:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767599.115243:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114735360:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775767599.115251:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767596/real 1775767599]  req@ffff9f1485746680 x1862022114734848/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767621 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767599.115263:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746680 x1862022114734848/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767621 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767599.115272:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767599.115275:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767599.115279:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114734848:192.168.202.110@tcp:250:kworker.0
00010000:00010000:1.0:1775767599.370729:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767600.138640:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767600.394773:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767601.162700:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775767601.162701:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767601.162714:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775767601.162720:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767601.162735:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4978
00000100:00080000:3.0:1775767601.162749:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767601.162759:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775767601.162817:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745500 x1862022114735872/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767601.162854:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767601.162859:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775767601.162861:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767601.162868:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4978
00000100:00080000:3.0:1775767601.162874:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767601.162880:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767601.162894:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775767601.162899:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114735872:192.168.202.110@tcp:250:kworker.0
00000100:00100000:3.0:1775767601.162925:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744e00 x1862022114736000/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767601.162942:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767601.162946:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775767601.162949:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767601.162955:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4978
00000100:00080000:3.0:1775767601.162960:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767601.162966:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767601.162972:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000800:00000100:2.0:1775767601.162974:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767601.163012:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744380 x1862022114736128/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767601.163025:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767601.163030:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775767601.163033:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767601.163040:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4978
00000100:00080000:3.0:1775767601.163045:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00100000:2.0:1775767601.163052:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767601/real 1775767601]  req@ffff9f1485745500 x1862022114735872/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767626 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767601.163053:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767601.163063:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775767601.163081:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745500 x1862022114735872/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767626 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767601.163093:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746a00 x1862022114736256/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767601.163094:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767601.163107:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767601.163119:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114735872:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775767601.163124:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767601.163129:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775767601.163132:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767601.163138:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4978
00000100:00080000:3.0:1775767601.163143:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767601.163149:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767601.163157:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775767601.163160:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114736000:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767601.163178:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744a80 x1862022114736384/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767601.163194:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767601.163203:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000800:00000100:2.0:1775767601.163204:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775767601.163208:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767601.163217:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4978
00000100:00080000:3.0:1775767601.163224:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:2.0:1775767601.163230:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767601/real 1775767601]  req@ffff9f1485744e00 x1862022114736000/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767626 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767601.163235:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767601.163248:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767601.163276:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747480 x1862022114736512/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775767601.163289:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744e00 x1862022114736000/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767626 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767601.163296:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767601.163310:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775767601.163317:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767601.163323:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767601.163329:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4978
00000100:00080000:2.0:1775767601.163336:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00080000:3.0:1775767601.163338:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:2.0:1775767601.163345:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114736000:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775767601.163351:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767601.163366:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775767601.163372:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114736128:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767601.163389:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745880 x1862022114736640/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767601.163401:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767601.163411:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000800:00000100:2.0:1775767601.163413:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775767601.163416:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767601.163425:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4978
00000100:00080000:3.0:1775767601.163432:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:2.0:1775767601.163439:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767601/real 1775767601]  req@ffff9f1485744380 x1862022114736128/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767626 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767601.163444:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775767601.163463:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744380 x1862022114736128/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767626 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767601.163471:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767601.163475:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767601.163478:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114736128:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767601.163484:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114736256:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775767601.163508:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767601.163519:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767601/real 1775767601]  req@ffff9f1485746a00 x1862022114736256/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767626 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767601.163532:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746a00 x1862022114736256/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767626 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767601.163547:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767601.163551:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767601.163586:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114736256:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775767601.163607:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114736384:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775767601.163629:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767601.163655:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767601/real 1775767601]  req@ffff9f1485744a80 x1862022114736384/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767626 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767601.163666:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744a80 x1862022114736384/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767626 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767601.163674:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767601.163677:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767601.163680:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114736384:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775767601.163687:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114736512:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775767601.163708:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767601.163720:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767601/real 1775767601]  req@ffff9f1485747480 x1862022114736512/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767626 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767601.163731:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747480 x1862022114736512/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767626 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767601.163739:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767601.163742:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767601.163745:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114736512:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767601.163751:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114736640:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775767601.163771:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767601.163782:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767601/real 1775767601]  req@ffff9f1485745880 x1862022114736640/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767626 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767601.163793:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745880 x1862022114736640/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767626 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767601.163799:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767601.163802:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767601.163805:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114736640:192.168.202.110@tcp:38:kworker.0
00000008:00100000:3.0:1775767601.164013:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775767601.164035:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746300 x1862022114736768/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775767601.164311:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114736768:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775767601.164346:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767601.164362:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767601/real 1775767601]  req@ffff9f1485746300 x1862022114736768/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767626 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767601.164376:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746300 x1862022114736768/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767626 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767601.164385:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767601.164389:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767601.164392:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114736768:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775767601.418815:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767602.186641:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767602.442773:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767603.210647:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767603.466726:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767604.234621:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767604.490672:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767605.258712:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767605.514773:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1775767606.282703:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:2.0:1775767606.282711:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775767606.282715:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775767606.282722:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767606.282733:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4983
00000100:00080000:3.0:1775767606.282753:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767606.282763:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775767606.282809:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747b80 x1862022114736896/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767606.282837:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767606.282840:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775767606.282842:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767606.282846:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4983
00000100:00080000:3.0:1775767606.282849:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767606.282851:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767606.282862:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767606.282874:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746300 x1862022114737024/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767606.282883:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767606.282886:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775767606.282887:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767606.282889:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4983
00000100:00080000:3.0:1775767606.282892:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767606.282894:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767606.282898:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767606.282905:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745880 x1862022114737152/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767606.282913:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:2.0:1775767606.282920:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114736896:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775767606.282927:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775767606.282944:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767606.282948:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4983
00000100:00080000:3.0:1775767606.282951:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767606.282957:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767606.282963:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775767606.282976:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747480 x1862022114737280/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767606.282983:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767606.282986:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775767606.282988:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767606.282992:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4983
00000100:00080000:3.0:1775767606.282994:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767606.282998:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767606.283001:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775767606.283007:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744a80 x1862022114737408/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767606.283018:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767606.283021:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775767606.283022:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767606.283027:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4983
00000100:00080000:3.0:1775767606.283029:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000800:00000100:2.0:1775767606.283035:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775767606.283045:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767606.283051:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767606.283071:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746a00 x1862022114737536/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767606.283083:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:2.0:1775767606.283088:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767606/real 1775767606]  req@ffff9f1485747b80 x1862022114736896/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767631 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767606.283090:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775767606.283093:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767606.283111:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4983
00000100:00080000:3.0:1775767606.283123:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767606.283128:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767606.283135:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775767606.283140:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747b80 x1862022114736896/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767631 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767606.283152:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744380 x1862022114737664/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767606.283161:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767606.283164:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767606.283171:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775767606.283173:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767606.283178:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:3.0:1775767606.283181:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4983
00000100:00080000:3.0:1775767606.283189:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:2.0:1775767606.283193:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114736896:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775767606.283199:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767606.283207:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775767606.283213:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114737024:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767606.283223:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744e00 x1862022114737792/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:2.0:1775767606.283249:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767606.283270:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767606/real 1775767606]  req@ffff9f1485746300 x1862022114737024/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767631 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767606.283288:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746300 x1862022114737024/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767631 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767606.283298:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767606.283302:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767606.283307:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114737024:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767606.283333:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114737152:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775767606.283356:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767606.283368:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767606/real 1775767606]  req@ffff9f1485745880 x1862022114737152/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767631 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767606.283381:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745880 x1862022114737152/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767631 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767606.283389:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767606.283393:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767606.283397:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114737152:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767606.283404:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114737280:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775767606.283426:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767606.283457:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767606/real 1775767606]  req@ffff9f1485747480 x1862022114737280/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767631 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767606.283473:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747480 x1862022114737280/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767631 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767606.283482:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767606.283486:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767606.283489:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114737280:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775767606.283497:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114737408:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775767606.283521:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767606.283534:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767606/real 1775767606]  req@ffff9f1485744a80 x1862022114737408/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767631 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767606.283547:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744a80 x1862022114737408/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767631 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767606.283585:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767606.283590:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767606.283594:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114737408:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775767606.283600:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114737536:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775767606.283624:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767606.283636:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767606/real 1775767606]  req@ffff9f1485746a00 x1862022114737536/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767631 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767606.283650:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746a00 x1862022114737536/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767631 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767606.283658:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767606.283662:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767606.283666:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114737536:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767606.283672:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114737664:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775767606.283692:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767606.283703:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767606/real 1775767606]  req@ffff9f1485744380 x1862022114737664/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767631 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767606.283715:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744380 x1862022114737664/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767631 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767606.283723:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767606.283727:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767606.283731:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114737664:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767606.283737:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114737792:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775767606.283759:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767606.283770:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767606/real 1775767606]  req@ffff9f1485744e00 x1862022114737792/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767631 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767606.283783:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744e00 x1862022114737792/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767631 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767606.283792:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767606.283795:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767606.283798:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114737792:192.168.202.110@tcp:8:kworker.0
00010000:00010000:1.0:1775767606.538773:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767607.306670:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767607.562790:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767608.330667:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767608.586700:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767609.354653:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767609.610739:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767610.378759:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767610.634750:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767611.402646:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:2.0:1775767611.402646:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767611.402655:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1775767611.402658:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767611.402666:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4988
00000100:00080000:2.0:1775767611.402676:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767611.402684:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775767611.402716:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1f80 x1862022114737920/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767611.402738:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767611.402741:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775767611.402742:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767611.402745:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4988
00000100:00080000:2.0:1775767611.402748:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767611.402751:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767611.402759:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767611.402774:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114737920:192.168.202.110@tcp:250:kworker.0
00000100:00100000:2.0:1775767611.402774:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2680 x1862022114738048/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767611.402790:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767611.402794:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775767611.402796:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767611.402800:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4988
00000100:00080000:2.0:1775767611.402805:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767611.402811:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767611.402817:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775767611.402831:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0700 x1862022114738176/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1775767611.402841:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:2.0:1775767611.402841:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767611.402846:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775767611.402848:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767611.402854:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4988
00000100:00080000:2.0:1775767611.402860:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767611.402864:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767611.402871:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775767611.402876:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767611/real 1775767611]  req@ffff9f14852d1f80 x1862022114737920/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767636 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767611.402903:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0a80 x1862022114738304/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767611.402930:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767611.402933:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1775767611.402935:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767611.402938:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4988
00000100:00100000:0.0:1775767611.402942:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1f80 x1862022114737920/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767636 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767611.402942:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767611.402950:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767611.402958:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00080000:0.0:1775767611.402960:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767611.402966:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767611.402971:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3800 x1862022114738432/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775767611.402972:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114737920:192.168.202.110@tcp:250:kworker.0
00000100:00080000:2.0:1775767611.402985:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767611.402987:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775767611.402988:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767611.402992:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4988
00000100:00080000:2.0:1775767611.402994:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767611.402997:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775767611.403001:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114738048:192.168.202.110@tcp:38:kworker.0
00000008:00100000:2.0:1775767611.403003:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775767611.403014:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2d80 x1862022114738560/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767611.403022:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767611.403026:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000800:00000100:0.0:1775767611.403029:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:2.0:1775767611.403030:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767611.403037:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4988
00000100:00080000:2.0:1775767611.403042:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1775767611.403043:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767611/real 1775767611]  req@ffff9f14852d2680 x1862022114738048/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767636 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767611.403051:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775767611.403059:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2680 x1862022114738048/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767636 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:2.0:1775767611.403061:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00080000:0.0:1775767611.403072:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767611.403076:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767611.403081:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114738048:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767611.403081:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1180 x1862022114738688/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775767611.403088:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114738176:192.168.202.110@tcp:38:kworker.0
00000100:00080000:2.0:1775767611.403098:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767611.403101:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775767611.403102:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767611.403106:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4988
00000100:00080000:2.0:1775767611.403110:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767611.403113:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767611.403118:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:0.0:1775767611.403119:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767611.403131:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767611/real 1775767611]  req@ffff9f14852d0700 x1862022114738176/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767636 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767611.403134:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2a00 x1862022114738816/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775767611.403157:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0700 x1862022114738176/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767636 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767611.403164:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767611.403166:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767611.403169:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114738176:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767611.403173:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114738304:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767611.403187:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767611.403192:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767611/real 1775767611]  req@ffff9f14852d0a80 x1862022114738304/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767636 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767611.403200:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0a80 x1862022114738304/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767636 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767611.403216:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767611.403218:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767611.403220:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114738304:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775767611.403224:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114738432:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767611.403235:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767611.403241:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767611/real 1775767611]  req@ffff9f14852d3800 x1862022114738432/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767636 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767611.403248:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3800 x1862022114738432/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767636 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767611.403253:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767611.403255:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767611.403256:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114738432:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775767611.403267:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114738560:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767611.403278:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767611.403284:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767611/real 1775767611]  req@ffff9f14852d2d80 x1862022114738560/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767636 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767611.403291:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2d80 x1862022114738560/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767636 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767611.403295:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767611.403297:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767611.403300:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114738560:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767611.403310:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114738688:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767611.403321:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767611.403327:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767611/real 1775767611]  req@ffff9f14852d1180 x1862022114738688/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767636 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767611.403334:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1180 x1862022114738688/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767636 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767611.403339:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767611.403341:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767611.403343:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114738688:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767611.403347:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114738816:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767611.403356:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767611.403361:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767611/real 1775767611]  req@ffff9f14852d2a00 x1862022114738816/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767636 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767611.403368:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2a00 x1862022114738816/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767636 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767611.403373:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767611.403375:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767611.403377:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114738816:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775767611.658698:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767612.426648:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767612.682717:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767613.450629:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775767613.706731:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767614.474664:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767614.730714:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767615.498648:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767615.754765:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767616.522628:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:2.0:1775767616.522634:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767616.522642:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1775767616.522646:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767616.522657:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4994
00000100:00080000:2.0:1775767616.522667:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767616.522673:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775767616.522711:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1500 x1862022114738944/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767616.522739:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767616.522742:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775767616.522744:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767616.522749:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4994
00000100:00080000:2.0:1775767616.522753:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767616.522758:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775767616.522765:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114738944:192.168.202.110@tcp:250:kworker.0
00000008:00100000:2.0:1775767616.522769:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775767616.522787:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0380 x1862022114739072/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767616.522800:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767616.522804:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775767616.522806:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767616.522810:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4994
00000100:00080000:2.0:1775767616.522815:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767616.522819:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767616.522825:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000800:00000100:0.0:1775767616.522844:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767616.522856:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0000 x1862022114739200/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767616.522867:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767616.522871:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775767616.522873:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767616.522879:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4994
00000100:00100000:0.0:1775767616.522881:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767616/real 1775767616]  req@ffff9f14852d1500 x1862022114738944/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767641 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767616.522885:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767616.522893:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767616.522899:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775767616.522901:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1500 x1862022114738944/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767641 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767616.522912:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767616.522916:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767616.522916:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3b80 x1862022114739328/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775767616.522923:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114738944:192.168.202.110@tcp:250:kworker.0
00000100:00080000:2.0:1775767616.522938:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767616.522942:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1775767616.522944:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775767616.522949:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114739072:192.168.202.110@tcp:38:kworker.0
00000100:00080000:2.0:1775767616.522952:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4994
00000100:00080000:2.0:1775767616.522959:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767616.522964:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767616.522970:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:0.0:1775767616.522986:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767616.522986:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0e00 x1862022114739456/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767616.523000:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1775767616.523006:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767616/real 1775767616]  req@ffff9f14852d0380 x1862022114739072/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767641 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767616.523006:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775767616.523010:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767616.523020:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4994
00000100:00080000:2.0:1775767616.523028:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1775767616.523033:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0380 x1862022114739072/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767641 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767616.523034:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767616.523043:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00080000:0.0:1775767616.523053:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767616.523060:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767616.523060:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1880 x1862022114739584/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775767616.523069:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114739072:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767616.523083:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114739200:192.168.202.110@tcp:38:kworker.0
00000100:00080000:2.0:1775767616.523084:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767616.523093:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775767616.523097:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767616.523102:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4994
00000100:00080000:2.0:1775767616.523107:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767616.523112:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767616.523117:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000800:00000100:0.0:1775767616.523119:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767616.523134:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3100 x1862022114739712/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775767616.523137:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767616/real 1775767616]  req@ffff9f14852d0000 x1862022114739200/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767641 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767616.523157:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1775767616.523164:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0000 x1862022114739200/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767641 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767616.523164:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775767616.523168:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767616.523178:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4994
00000100:00080000:2.0:1775767616.523185:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767616.523186:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767616.523191:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00080000:0.0:1775767616.523196:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000008:00100000:2.0:1775767616.523199:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775767616.523202:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114739200:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767616.523212:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114739328:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775767616.523215:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2300 x1862022114739840/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1775767616.523243:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767616.523257:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767616/real 1775767616]  req@ffff9f14852d3b80 x1862022114739328/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767641 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767616.523270:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3b80 x1862022114739328/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767641 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767616.523280:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767616.523283:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767616.523286:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114739328:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775767616.523303:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114739456:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767616.523325:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767616.523337:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767616/real 1775767616]  req@ffff9f14852d0e00 x1862022114739456/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767641 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767616.523357:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0e00 x1862022114739456/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767641 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767616.523366:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767616.523369:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767616.523373:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114739456:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775767616.523379:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114739584:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767616.523398:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767616.523408:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767616/real 1775767616]  req@ffff9f14852d1880 x1862022114739584/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767641 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767616.523421:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1880 x1862022114739584/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767641 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767616.523430:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767616.523433:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767616.523437:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114739584:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767616.523442:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114739712:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767616.523460:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767616.523470:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767616/real 1775767616]  req@ffff9f14852d3100 x1862022114739712/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767641 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767616.523482:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3100 x1862022114739712/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767641 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767616.523497:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767616.523500:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767616.523503:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114739712:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767616.523509:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114739840:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767616.523527:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767616.523537:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767616/real 1775767616]  req@ffff9f14852d2300 x1862022114739840/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767641 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767616.523549:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2300 x1862022114739840/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767641 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767616.523579:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767616.523583:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767616.523586:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114739840:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775767616.778775:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767617.546690:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767617.802755:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767618.570684:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775767618.826707:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767619.594665:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767619.850711:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767620.618640:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775767620.874711:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1775767621.642684:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767621.642696:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775767621.642701:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00100000:2.0:1775767621.642709:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775767621.642712:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 4999
00000100:00080000:3.0:1775767621.642727:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767621.642734:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775767621.642795:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745500 x1862022114739968/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767621.642827:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767621.642831:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775767621.642832:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767621.642836:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4999
00000100:00080000:3.0:1775767621.642840:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767621.642844:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767621.642855:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767621.642870:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746680 x1862022114740096/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775767621.642874:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114739968:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775767621.642912:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767621.642917:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775767621.642919:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767621.642927:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4999
00000100:00080000:3.0:1775767621.642932:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767621.642937:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767621.642945:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767621.642957:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745c00 x1862022114740224/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1775767621.642968:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775767621.642982:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767621.642985:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775767621.642988:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767621.642992:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4999
00000100:00080000:3.0:1775767621.642997:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767621.643000:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767621.643005:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775767621.643020:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745180 x1862022114740352/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775767621.643022:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767621/real 1775767621]  req@ffff9f1485745500 x1862022114739968/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767646 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767621.643036:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767621.643051:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775767621.643057:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767621.643069:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 4999
00000100:00080000:3.0:1775767621.643074:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767621.643079:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775767621.643083:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745500 x1862022114739968/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767646 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:3.0:1775767621.643084:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775767621.643099:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747100 x1862022114740480/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767621.643101:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767621.643118:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767621.643119:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:3.0:1775767621.643131:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00100000:0.0:1775767621.643135:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114739968:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775767621.643135:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767621.643148:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4999
00000100:00080000:3.0:1775767621.643153:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767621.643159:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767621.643163:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767621.643172:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114740096:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767621.643179:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744000 x1862022114740608/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767621.643191:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767621.643195:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775767621.643197:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767621.643205:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4999
00000800:00000100:0.0:1775767621.643207:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775767621.643215:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767621.643226:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775767621.643227:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767621/real 1775767621]  req@ffff9f1485746680 x1862022114740096/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767646 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:3.0:1775767621.643236:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767621.643247:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746680 x1862022114740096/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767646 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767621.643255:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767621.643258:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767621.643263:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747800 x1862022114740736/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775767621.643265:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114740096:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767621.643288:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114740224:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775767621.643314:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000800:00000100:0.0:1775767621.643320:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775767621.643325:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775767621.643329:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775767621.643334:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767621/real 1775767621]  req@ffff9f1485745c00 x1862022114740224/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767646 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767621.643341:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 4999
00000100:00100000:0.0:1775767621.643359:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745c00 x1862022114740224/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767646 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767621.643359:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767621.643372:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00080000:0.0:1775767621.643386:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000008:00100000:3.0:1775767621.643388:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00080000:0.0:1775767621.643392:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767621.643400:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114740224:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767621.643406:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114740352:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775767621.643410:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745f80 x1862022114740864/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1775767621.643436:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767621.643446:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767621/real 1775767621]  req@ffff9f1485745180 x1862022114740352/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767646 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767621.643457:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745180 x1862022114740352/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767646 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767621.643463:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767621.643467:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767621.643470:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114740352:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775767621.643474:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114740480:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767621.643484:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767621.643490:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767621/real 1775767621]  req@ffff9f1485747100 x1862022114740480/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767646 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767621.643497:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747100 x1862022114740480/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767646 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767621.643502:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767621.643504:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767621.643506:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114740480:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775767621.643518:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114740608:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767621.643527:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767621.643533:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767621/real 1775767621]  req@ffff9f1485744000 x1862022114740608/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767646 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767621.643545:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744000 x1862022114740608/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767646 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767621.643550:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767621.643571:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767621.643574:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114740608:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767621.643578:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114740736:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767621.643590:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767621.643597:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767621/real 1775767621]  req@ffff9f1485747800 x1862022114740736/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767646 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767621.643605:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747800 x1862022114740736/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767646 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767621.643610:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767621.643612:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767621.643614:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114740736:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767621.643617:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114740864:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767621.643627:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767621.643632:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767621/real 1775767621]  req@ffff9f1485745f80 x1862022114740864/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767646 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767621.643639:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745f80 x1862022114740864/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767646 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767621.643648:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767621.643650:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767621.643652:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114740864:192.168.202.110@tcp:8:kworker.0
00010000:00010000:3.0:1775767621.898819:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767622.666705:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767622.922731:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767623.690632:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767623.946791:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767624.714680:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767624.970827:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767625.738685:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775767625.994714:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767626.762651:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:0.0:1775767626.762666:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767626.762673:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1775767626.762678:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767626.762689:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5004
00000100:00080000:0.0:1775767626.762700:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767626.762708:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775767626.762765:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0a80 x1862022114740992/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767626.762795:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767626.762799:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775767626.762801:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767626.762805:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5004
00000100:00080000:0.0:1775767626.762809:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767626.762815:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767626.762826:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767626.762845:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d2300 x1862022114741120/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775767626.762845:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114740992:192.168.202.110@tcp:250:kworker.0
00000100:00080000:0.0:1775767626.762865:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767626.762869:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:0.0:1775767626.762872:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767626.762878:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5004
00000100:00080000:0.0:1775767626.762882:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767626.762887:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767626.762893:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767626.762905:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d2a00 x1862022114741248/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767626.762913:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767626.762916:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775767626.762919:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000800:00000100:3.0:1775767626.762921:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:0.0:1775767626.762925:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5004
00000100:00080000:0.0:1775767626.762928:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767626.762932:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767626.762936:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775767626.762949:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3800 x1862022114741376/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767626.762958:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767626.762961:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1775767626.762963:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767626.762967:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5004
00000100:00080000:0.0:1775767626.762970:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767626.762974:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767626.762978:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775767626.762988:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d1500 x1862022114741504/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767626.762996:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767626.763000:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775767626.763002:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767626.763006:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5004
00000100:00080000:0.0:1775767626.763011:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767626.763015:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767626.763021:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767626.763021:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767626/real 1775767626]  req@ffff9f14a04d0a80 x1862022114740992/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767651 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767626.763039:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0000 x1862022114741632/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767626.763047:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767626.763050:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:0.0:1775767626.763051:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767626.763055:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5004
00000100:00080000:0.0:1775767626.763058:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767626.763061:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767626.763066:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767626.763076:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0e00 x1862022114741760/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767626.763083:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767626.763087:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775767626.763088:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767626.763093:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5004
00000100:00080000:0.0:1775767626.763096:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767626.763101:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767626.763105:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775767626.763117:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3100 x1862022114741888/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775767626.763151:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0a80 x1862022114740992/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767651 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767626.763160:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767626.763166:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767626.763173:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114740992:192.168.202.110@tcp:250:kworker.0
00000100:00100000:3.0:1775767626.763278:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114741120:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775767626.763302:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767626.763314:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767626/real 1775767626]  req@ffff9f14a04d2300 x1862022114741120/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767651 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767626.763323:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d2300 x1862022114741120/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767651 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767626.763329:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767626.763331:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767626.763334:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114741120:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767626.763338:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114741248:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775767626.763349:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767626.763355:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767626/real 1775767626]  req@ffff9f14a04d2a00 x1862022114741248/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767651 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767626.763362:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d2a00 x1862022114741248/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767651 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767626.763367:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767626.763369:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767626.763372:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114741248:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767626.763378:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114741376:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775767626.763397:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767626.763407:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767626/real 1775767626]  req@ffff9f14a04d3800 x1862022114741376/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767651 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767626.763420:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3800 x1862022114741376/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767651 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767626.763429:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767626.763432:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767626.763436:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114741376:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775767626.763442:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114741504:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775767626.763463:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767626.763473:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767626/real 1775767626]  req@ffff9f14a04d1500 x1862022114741504/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767651 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767626.763485:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1500 x1862022114741504/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767651 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767626.763493:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767626.763496:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767626.763499:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114741504:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775767626.763505:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114741632:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775767626.763519:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767626.763527:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767626/real 1775767626]  req@ffff9f14a04d0000 x1862022114741632/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767651 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767626.763539:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0000 x1862022114741632/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767651 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767626.763547:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767626.763550:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767626.763583:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114741632:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767626.763591:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114741760:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775767626.763616:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767626.763635:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767626/real 1775767626]  req@ffff9f14a04d0e00 x1862022114741760/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767651 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767626.763649:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0e00 x1862022114741760/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767651 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767626.763658:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767626.763661:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767626.763664:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114741760:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767626.763670:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114741888:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775767626.763694:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767626.763706:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767626/real 1775767626]  req@ffff9f14a04d3100 x1862022114741888/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767651 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767626.763719:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3100 x1862022114741888/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767651 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767626.763728:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767626.763748:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767626.763752:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114741888:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775767627.018749:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767627.786701:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767628.042743:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767628.810657:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767629.066749:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767629.834670:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767630.090746:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767630.858730:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767631.114752:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767631.882699:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775767631.882719:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767631.882729:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775767631.882735:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767631.882746:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5009
00000100:00080000:3.0:1775767631.882756:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767631.882763:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775767631.882804:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744700 x1862022114742016/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767631.882835:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767631.882838:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775767631.882840:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767631.882844:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5009
00000100:00080000:3.0:1775767631.882847:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767631.882850:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767631.882859:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767631.882873:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745f80 x1862022114742144/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767631.882883:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767631.882888:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775767631.882891:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775767631.882892:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114742016:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775767631.882896:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5009
00000100:00080000:3.0:1775767631.882903:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767631.882908:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767631.882912:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767631.882921:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747800 x1862022114742272/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767631.882930:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767631.882934:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775767631.882936:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767631.882939:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5009
00000100:00080000:3.0:1775767631.882943:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767631.882947:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767631.882950:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775767631.882959:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744000 x1862022114742400/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767631.882967:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000800:00000100:0.0:1775767631.882974:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775767631.882975:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775767631.882979:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767631.882983:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5009
00000100:00080000:3.0:1775767631.882987:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767631.882991:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767631.882996:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775767631.883006:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747100 x1862022114742528/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767631.883018:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767631.883023:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775767631.883026:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775767631.883029:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767631/real 1775767631]  req@ffff9f1485744700 x1862022114742016/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767656 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767631.883035:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5009
00000100:00080000:3.0:1775767631.883046:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767631.883056:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767631.883064:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767631.883067:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744700 x1862022114742016/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767656 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767631.883081:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00100000:3.0:1775767631.883087:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745180 x1862022114742656/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767631.883089:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767631.883098:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114742016:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775767631.883108:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767631.883111:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775767631.883114:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767631.883134:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5009
00000100:00080000:3.0:1775767631.883139:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767631.883144:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775767631.883150:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114742144:192.168.202.110@tcp:38:kworker.0
00000008:00100000:3.0:1775767631.883150:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767631.883167:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745c00 x1862022114742784/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767631.883178:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767631.883181:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775767631.883183:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767631.883187:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5009
00000100:00080000:3.0:1775767631.883191:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000800:00000100:0.0:1775767631.883194:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775767631.883199:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767631.883205:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775767631.883218:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767631/real 1775767631]  req@ffff9f1485745f80 x1862022114742144/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767656 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767631.883219:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746680 x1862022114742912/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775767631.883252:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745f80 x1862022114742144/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767656 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767631.883262:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767631.883266:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767631.883270:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114742144:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767631.883276:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114742272:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767631.883294:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767631.883307:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767631/real 1775767631]  req@ffff9f1485747800 x1862022114742272/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767656 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767631.883324:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747800 x1862022114742272/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767656 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767631.883335:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767631.883339:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767631.883344:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114742272:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767631.883353:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114742400:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767631.883380:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767631.883392:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767631/real 1775767631]  req@ffff9f1485744000 x1862022114742400/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767656 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767631.883403:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744000 x1862022114742400/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767656 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767631.883410:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767631.883413:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767631.883415:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114742400:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775767631.883420:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114742528:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767631.883436:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767631.883443:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767631/real 1775767631]  req@ffff9f1485747100 x1862022114742528/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767656 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767631.883452:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747100 x1862022114742528/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767656 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767631.883459:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767631.883461:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767631.883463:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114742528:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775767631.883468:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114742656:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767631.883480:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767631.883487:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767631/real 1775767631]  req@ffff9f1485745180 x1862022114742656/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767656 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767631.883496:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745180 x1862022114742656/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767656 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767631.883503:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767631.883505:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767631.883507:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114742656:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767631.883523:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114742784:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767631.883536:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767631.883543:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767631/real 1775767631]  req@ffff9f1485745c00 x1862022114742784/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767656 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767631.883584:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745c00 x1862022114742784/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767656 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767631.883592:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767631.883594:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767631.883597:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114742784:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767631.883602:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114742912:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767631.883618:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767631.883627:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767631/real 1775767631]  req@ffff9f1485746680 x1862022114742912/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767656 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767631.883637:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746680 x1862022114742912/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767656 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767631.883643:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767631.883646:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767631.883649:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114742912:192.168.202.110@tcp:8:kworker.0
00010000:00010000:0.0:1775767632.138765:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767632.906625:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767633.162731:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767633.930690:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767634.186719:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767634.954642:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767635.210725:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767635.978684:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767636.234794:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767637.002680:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:1.0:1775767637.002713:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767637.002721:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1775767637.002725:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767637.002735:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5014
00000100:00080000:1.0:1775767637.002746:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767637.002754:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775767637.002793:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ffb80 x1862022114743040/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767637.002822:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767637.002825:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775767637.002828:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767637.002832:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5014
00000100:00080000:1.0:1775767637.002836:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767637.002840:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767637.002851:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767637.002866:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fed80 x1862022114743168/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767637.002878:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767637.002882:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775767637.002885:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775767637.002888:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ffb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114743040:192.168.202.110@tcp:250:kworker.0
00000100:00080000:1.0:1775767637.002892:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5014
00000100:00080000:1.0:1775767637.002900:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767637.002904:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767637.002911:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767637.002924:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff800 x1862022114743296/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767637.002935:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767637.002939:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775767637.002941:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767637.002945:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5014
00000100:00080000:1.0:1775767637.002950:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767637.002954:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767637.002959:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:0.0:1775767637.002960:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767637.002976:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fca80 x1862022114743424/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767637.002986:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767637.002990:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1775767637.002992:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767637.002996:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5014
00000100:00080000:1.0:1775767637.003001:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767637.003005:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775767637.003010:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767637/real 1775767637]  req@ffff9f14ab4ffb80 x1862022114743040/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767662 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:1.0:1775767637.003013:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775767637.003032:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff100 x1862022114743552/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775767637.003038:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ffb80 x1862022114743040/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767662 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767637.003058:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767637.003065:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:1.0:1775767637.003065:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767637.003073:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00100000:0.0:1775767637.003074:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ffb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114743040:192.168.202.110@tcp:250:kworker.0
00000100:00080000:1.0:1775767637.003077:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767637.003086:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5014
00000100:00080000:1.0:1775767637.003091:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767637.003095:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767637.003102:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767637.003106:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114743168:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775767637.003117:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdc00 x1862022114743680/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767637.003130:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767637.003134:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775767637.003138:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000800:00000100:0.0:1775767637.003139:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:1.0:1775767637.003148:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5014
00000100:00080000:1.0:1775767637.003153:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767637.003158:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775767637.003161:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767637/real 1775767637]  req@ffff9f14ab4fed80 x1862022114743168/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767662 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:1.0:1775767637.003165:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767637.003183:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fce00 x1862022114743808/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775767637.003186:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fed80 x1862022114743168/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767662 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767637.003211:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767637.003213:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767637.003218:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775767637.003221:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00080000:1.0:1775767637.003222:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775767637.003230:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114743168:192.168.202.110@tcp:38:kworker.0
00000100:00080000:1.0:1775767637.003231:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5014
00000100:00080000:1.0:1775767637.003240:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767637.003246:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767637.003252:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775767637.003254:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114743296:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775767637.003268:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fc380 x1862022114743936/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1775767637.003284:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767637.003299:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767637/real 1775767637]  req@ffff9f14ab4ff800 x1862022114743296/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767662 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767637.003313:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff800 x1862022114743296/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767662 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767637.003322:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767637.003326:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767637.003329:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114743296:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767637.003337:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114743424:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767637.003359:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767637.003370:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767637/real 1775767637]  req@ffff9f14ab4fca80 x1862022114743424/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767662 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767637.003382:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fca80 x1862022114743424/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767662 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767637.003391:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767637.003394:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767637.003397:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114743424:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775767637.003403:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114743552:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767637.003423:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767637.003435:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767637/real 1775767637]  req@ffff9f14ab4ff100 x1862022114743552/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767662 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767637.003447:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff100 x1862022114743552/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767662 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767637.003455:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767637.003458:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767637.003461:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114743552:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775767637.003479:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114743680:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767637.003500:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767637.003513:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767637/real 1775767637]  req@ffff9f14ab4fdc00 x1862022114743680/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767662 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767637.003525:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdc00 x1862022114743680/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767662 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767637.003532:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767637.003535:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767637.003538:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114743680:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767637.003544:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114743808:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767637.003596:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767637.003608:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767637/real 1775767637]  req@ffff9f14ab4fce00 x1862022114743808/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767662 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767637.003620:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fce00 x1862022114743808/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767662 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767637.003628:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767637.003652:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767637.003656:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114743808:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767637.003662:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114743936:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767637.003683:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767637.003694:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767637/real 1775767637]  req@ffff9f14ab4fc380 x1862022114743936/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767662 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767637.003706:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fc380 x1862022114743936/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767662 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767637.003721:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767637.003724:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767637.003727:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114743936:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775767637.258780:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767638.026673:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767638.282739:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767639.050710:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:0.0:1775767639.306694:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767640.074654:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767640.330744:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767641.098690:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767641.354736:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1775767642.122675:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:2.0:1775767642.122676:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775767642.122684:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775767642.122688:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767642.122696:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5019
00000100:00080000:3.0:1775767642.122705:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767642.122712:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775767642.122756:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745500 x1862022114744064/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767642.122778:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767642.122782:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775767642.122783:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767642.122788:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5019
00000100:00080000:3.0:1775767642.122791:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767642.122796:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767642.122806:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767642.122818:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744e00 x1862022114744192/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775767642.122827:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114744064:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775767642.122829:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767642.122832:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775767642.122834:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767642.122837:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5019
00000100:00080000:3.0:1775767642.122842:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767642.122845:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767642.122849:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767642.122858:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744380 x1862022114744320/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767642.122865:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767642.122868:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775767642.122869:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767642.122872:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5019
00000100:00080000:3.0:1775767642.122875:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767642.122878:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767642.122882:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:0.0:1775767642.122885:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767642.122890:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746a00 x1862022114744448/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767642.122898:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767642.122901:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775767642.122902:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767642.122906:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5019
00000100:00080000:3.0:1775767642.122908:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767642.122911:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767642.122916:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775767642.122923:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767642/real 1775767642]  req@ffff9f1485745500 x1862022114744064/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767667 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767642.122924:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744a80 x1862022114744576/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767642.122934:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767642.122937:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775767642.122939:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775767642.122940:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745500 x1862022114744064/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767667 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767642.122942:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5019
00000100:00080000:3.0:1775767642.122946:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767642.122950:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00080000:0.0:1775767642.122951:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000008:00100000:3.0:1775767642.122954:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00080000:0.0:1775767642.122955:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767642.122959:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114744064:192.168.202.110@tcp:250:kworker.0
00000100:00100000:3.0:1775767642.122963:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747480 x1862022114744704/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767642.122971:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767642.122974:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775767642.122976:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775767642.122979:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114744192:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775767642.122979:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5019
00000100:00080000:3.0:1775767642.122982:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767642.122985:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767642.122990:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767642.122999:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745880 x1862022114744832/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1775767642.123000:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775767642.123008:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1775767642.123011:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767642/real 1775767642]  req@ffff9f1485744e00 x1862022114744192/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767667 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767642.123012:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775767642.123013:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767642.123017:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5019
00000100:00080000:3.0:1775767642.123021:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1775767642.123024:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744e00 x1862022114744192/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767667 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767642.123025:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767642.123030:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00080000:0.0:1775767642.123032:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767642.123035:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767642.123038:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746300 x1862022114744960/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775767642.123039:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114744192:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767642.123045:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114744320:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767642.123063:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767642.123074:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767642/real 1775767642]  req@ffff9f1485744380 x1862022114744320/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767667 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767642.123086:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744380 x1862022114744320/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767667 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767642.123102:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767642.123104:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767642.123106:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114744320:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767642.123111:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114744448:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767642.123127:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767642.123134:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767642/real 1775767642]  req@ffff9f1485746a00 x1862022114744448/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767667 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767642.123142:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746a00 x1862022114744448/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767667 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767642.123147:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767642.123149:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767642.123151:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114744448:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775767642.123154:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114744576:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767642.123164:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767642.123169:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767642/real 1775767642]  req@ffff9f1485744a80 x1862022114744576/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767667 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767642.123176:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744a80 x1862022114744576/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767667 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767642.123181:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767642.123183:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767642.123185:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114744576:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775767642.123188:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114744704:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767642.123197:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767642.123203:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767642/real 1775767642]  req@ffff9f1485747480 x1862022114744704/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767667 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767642.123210:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747480 x1862022114744704/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767667 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767642.123215:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767642.123216:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767642.123218:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114744704:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767642.123229:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114744832:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767642.123238:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767642.123244:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767642/real 1775767642]  req@ffff9f1485745880 x1862022114744832/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767667 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767642.123250:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745880 x1862022114744832/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767667 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767642.123256:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767642.123265:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767642.123268:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114744832:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767642.123271:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114744960:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767642.123282:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767642.123288:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767642/real 1775767642]  req@ffff9f1485746300 x1862022114744960/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767667 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767642.123295:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746300 x1862022114744960/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767667 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767642.123300:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767642.123302:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767642.123304:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114744960:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775767642.378751:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767643.146651:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767643.402694:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775767644.170660:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767644.426727:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775767645.194629:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767645.450686:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767646.218651:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:0.0:1775767646.474683:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1775767647.242666:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767647.242682:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775767647.242686:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767647.242696:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5024
00000100:00100000:2.0:1775767647.242704:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775767647.242708:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767647.242717:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775767647.242760:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747b80 x1862022114745088/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767647.242783:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767647.242786:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775767647.242787:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767647.242791:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5024
00000100:00080000:3.0:1775767647.242793:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767647.242795:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767647.242804:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767647.242815:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746300 x1862022114745216/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767647.242823:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767647.242827:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775767647.242830:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775767647.242832:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114745088:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775767647.242835:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5024
00000100:00080000:3.0:1775767647.242840:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767647.242843:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767647.242846:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767647.242853:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745880 x1862022114745344/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767647.242859:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767647.242861:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775767647.242862:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767647.242865:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5024
00000100:00080000:3.0:1775767647.242868:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767647.242870:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767647.242874:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775767647.242880:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747480 x1862022114745472/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767647.242886:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767647.242888:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775767647.242889:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767647.242891:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5024
00000100:00080000:3.0:1775767647.242894:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767647.242897:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767647.242901:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:0.0:1775767647.242911:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767647.242916:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744a80 x1862022114745600/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767647.242924:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767647.242927:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775767647.242928:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767647.242931:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5024
00000100:00080000:3.0:1775767647.242934:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767647.242937:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767647.242940:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767647.242949:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746a00 x1862022114745728/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767647.242956:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1775767647.242957:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767647/real 1775767647]  req@ffff9f1485747b80 x1862022114745088/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767672 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767647.242960:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775767647.242963:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767647.242970:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5024
00000100:00080000:3.0:1775767647.242980:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767647.242984:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767647.242988:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767647.242991:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747b80 x1862022114745088/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767672 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767647.242998:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744380 x1862022114745856/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767647.243016:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767647.243024:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767647.243029:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:3.0:1775767647.243029:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775767647.243032:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775767647.243037:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114745088:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775767647.243039:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5024
00000100:00080000:3.0:1775767647.243047:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767647.243050:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767647.243053:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775767647.243060:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744e00 x1862022114745984/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775767647.243066:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114745216:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767647.243090:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767647.243101:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767647/real 1775767647]  req@ffff9f1485746300 x1862022114745216/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767672 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767647.243109:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746300 x1862022114745216/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767672 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767647.243115:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767647.243117:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767647.243119:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114745216:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767647.243123:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114745344:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767647.243133:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767647.243139:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767647/real 1775767647]  req@ffff9f1485745880 x1862022114745344/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767672 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767647.243146:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745880 x1862022114745344/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767672 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767647.243151:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767647.243152:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767647.243154:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114745344:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767647.243158:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114745472:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767647.243168:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767647.243173:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767647/real 1775767647]  req@ffff9f1485747480 x1862022114745472/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767672 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767647.243180:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747480 x1862022114745472/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767672 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767647.243185:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767647.243187:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767647.243188:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114745472:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775767647.243192:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114745600:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767647.243200:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767647.243205:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767647/real 1775767647]  req@ffff9f1485744a80 x1862022114745600/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767672 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767647.243212:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744a80 x1862022114745600/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767672 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767647.243217:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767647.243218:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767647.243220:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114745600:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775767647.243223:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114745728:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767647.243232:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767647.243237:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767647/real 1775767647]  req@ffff9f1485746a00 x1862022114745728/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767672 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767647.243244:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746a00 x1862022114745728/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767672 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767647.243249:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767647.243251:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767647.243253:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114745728:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767647.243256:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114745856:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767647.243265:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767647.243270:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767647/real 1775767647]  req@ffff9f1485744380 x1862022114745856/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767672 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767647.243276:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744380 x1862022114745856/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767672 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767647.243281:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767647.243283:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767647.243285:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114745856:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767647.243297:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114745984:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767647.243306:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767647.243311:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767647/real 1775767647]  req@ffff9f1485744e00 x1862022114745984/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767672 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767647.243318:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744e00 x1862022114745984/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767672 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767647.243323:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767647.243325:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767647.243327:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114745984:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775767647.498697:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767648.266636:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767648.522739:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767649.290747:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767649.546743:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767650.314648:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767650.570764:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767651.338686:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767651.594731:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767652.362719:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775767652.362737:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767652.362747:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775767652.362751:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767652.362762:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5029
00000100:00080000:3.0:1775767652.362773:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767652.362782:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775767652.362822:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745500 x1862022114746112/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767652.362852:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767652.362855:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775767652.362857:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767652.362862:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5029
00000100:00080000:3.0:1775767652.362866:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767652.362870:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767652.362880:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767652.362893:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746680 x1862022114746240/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767652.362904:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767652.362908:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775767652.362910:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767652.362915:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5029
00000100:00080000:3.0:1775767652.362920:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767652.362924:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767652.362929:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767652.362941:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745c00 x1862022114746368/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775767652.362944:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114746112:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775767652.362960:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767652.362963:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775767652.362965:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767652.362971:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5029
00000100:00080000:3.0:1775767652.362974:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767652.362979:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767652.362985:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:0.0:1775767652.363006:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767652.363028:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745180 x1862022114746496/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767652.363040:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1775767652.363044:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767652/real 1775767652]  req@ffff9f1485745500 x1862022114746112/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767677 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767652.363047:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775767652.363051:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767652.363062:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5029
00000100:00100000:0.0:1775767652.363067:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745500 x1862022114746112/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767677 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767652.363068:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767652.363077:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00080000:0.0:1775767652.363081:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000008:00100000:3.0:1775767652.363084:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00080000:0.0:1775767652.363087:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767652.363092:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114746112:192.168.202.110@tcp:250:kworker.0
00000100:00100000:0.0:1775767652.363099:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114746240:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767652.363103:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747100 x1862022114746624/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767652.363116:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000800:00000100:0.0:1775767652.363119:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775767652.363124:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775767652.363126:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775767652.363130:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767652/real 1775767652]  req@ffff9f1485746680 x1862022114746240/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767677 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767652.363133:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5029
00000100:00080000:3.0:1775767652.363142:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1775767652.363147:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746680 x1862022114746240/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767677 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767652.363150:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767652.363159:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00080000:0.0:1775767652.363160:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767652.363167:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767652.363169:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114746240:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767652.363179:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744000 x1862022114746752/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775767652.363189:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114746368:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775767652.363191:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767652.363198:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775767652.363200:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000800:00000100:0.0:1775767652.363207:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775767652.363207:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5029
00000100:00080000:3.0:1775767652.363216:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1775767652.363220:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767652/real 1775767652]  req@ffff9f1485745c00 x1862022114746368/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767677 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767652.363224:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767652.363234:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767652.363238:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745c00 x1862022114746368/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767677 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767652.363244:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767652.363247:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767652.363251:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747800 x1862022114746880/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775767652.363253:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114746368:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767652.363263:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114746496:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767652.363280:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775767652.363282:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1775767652.363288:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767652/real 1775767652]  req@ffff9f1485745180 x1862022114746496/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767677 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767652.363289:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775767652.363293:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767652.363303:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5029
00000100:00100000:0.0:1775767652.363307:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745180 x1862022114746496/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767677 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767652.363310:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767652.363317:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767652.363319:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00080000:0.0:1775767652.363324:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767652.363327:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114746496:192.168.202.110@tcp:8:kworker.0
00000008:00100000:3.0:1775767652.363328:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775767652.363332:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114746624:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767652.363344:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767652.363351:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767652/real 1775767652]  req@ffff9f1485747100 x1862022114746624/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767677 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767652.363359:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745f80 x1862022114747008/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775767652.363362:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747100 x1862022114746624/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767677 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767652.363390:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767652.363393:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767652.363396:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114746624:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775767652.363407:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114746752:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767652.363428:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767652.363438:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767652/real 1775767652]  req@ffff9f1485744000 x1862022114746752/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767677 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767652.363451:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744000 x1862022114746752/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767677 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767652.363460:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767652.363463:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767652.363467:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114746752:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767652.363473:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114746880:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767652.363584:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767652.363595:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767652/real 1775767652]  req@ffff9f1485747800 x1862022114746880/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767677 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767652.363603:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747800 x1862022114746880/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767677 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767652.363609:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767652.363611:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767652.363613:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114746880:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767652.363618:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114747008:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767652.363631:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767652.363637:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767652/real 1775767652]  req@ffff9f1485745f80 x1862022114747008/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767677 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767652.363644:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745f80 x1862022114747008/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767677 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767652.363649:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767652.363651:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767652.363652:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114747008:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775767652.618727:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767653.386672:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:0.0:1775767653.642672:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767654.410638:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767654.666730:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767655.434646:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767655.690791:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767656.458664:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767656.714738:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767657.482666:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:1.0:1775767657.482674:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767657.482681:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1775767657.482685:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767657.482696:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5034
00000100:00080000:1.0:1775767657.482708:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767657.482715:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775767657.482752:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdf80 x1862022114747136/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767657.482776:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767657.482779:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775767657.482780:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767657.482783:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5034
00000100:00080000:1.0:1775767657.482786:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767657.482789:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767657.482796:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767657.482810:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fe300 x1862022114747264/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767657.482818:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767657.482820:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775767657.482822:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767657.482824:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5034
00000100:00080000:1.0:1775767657.482827:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767657.482832:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775767657.482833:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114747136:192.168.202.110@tcp:250:kworker.0
00000008:00100000:1.0:1775767657.482835:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767657.482845:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fd880 x1862022114747392/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767657.482851:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767657.482854:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775767657.482855:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767657.482858:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5034
00000100:00080000:1.0:1775767657.482859:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767657.482862:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767657.482865:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775767657.482872:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fea00 x1862022114747520/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767657.482883:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767657.482885:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1775767657.482886:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767657.482889:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5034
00000100:00080000:1.0:1775767657.482891:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000800:00000100:0.0:1775767657.482896:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:1.0:1775767657.482897:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767657.482902:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775767657.482910:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fc380 x1862022114747648/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767657.482917:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767657.482920:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775767657.482921:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767657.482923:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5034
00000100:00080000:1.0:1775767657.482926:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767657.482929:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767657.482933:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767657.482942:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767657/real 1775767657]  req@ffff9f14ab4fdf80 x1862022114747136/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767682 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767657.482942:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fce00 x1862022114747776/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767657.482969:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767657.482971:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775767657.482972:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767657.482976:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5034
00000100:00080000:1.0:1775767657.482978:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1775767657.482981:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdf80 x1862022114747136/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767682 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767657.482984:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767657.482990:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767657.482996:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdc00 x1862022114747904/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767657.483012:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767657.483013:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767657.483019:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775767657.483021:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:1.0:1775767657.483021:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775767657.483026:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114747136:192.168.202.110@tcp:250:kworker.0
00000100:00080000:1.0:1775767657.483032:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5034
00000100:00080000:1.0:1775767657.483035:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1775767657.483040:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114747264:192.168.202.110@tcp:38:kworker.0
00000100:00080000:1.0:1775767657.483040:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767657.483045:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775767657.483055:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff100 x1862022114748032/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1775767657.483086:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767657.483099:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767657/real 1775767657]  req@ffff9f14ab4fe300 x1862022114747264/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767682 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767657.483110:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fe300 x1862022114747264/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767682 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767657.483117:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767657.483119:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767657.483121:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114747264:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767657.483140:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114747392:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767657.483154:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767657.483161:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767657/real 1775767657]  req@ffff9f14ab4fd880 x1862022114747392/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767682 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767657.483168:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fd880 x1862022114747392/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767682 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767657.483174:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767657.483176:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767657.483178:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114747392:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767657.483181:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114747520:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767657.483192:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767657.483197:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767657/real 1775767657]  req@ffff9f14ab4fea00 x1862022114747520/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767682 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767657.483209:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fea00 x1862022114747520/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767682 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767657.483214:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767657.483216:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767657.483218:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114747520:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775767657.483221:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114747648:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767657.483230:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767657.483235:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767657/real 1775767657]  req@ffff9f14ab4fc380 x1862022114747648/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767682 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767657.483242:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fc380 x1862022114747648/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767682 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767657.483246:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767657.483248:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767657.483250:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114747648:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775767657.483253:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114747776:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767657.483262:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767657.483267:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767657/real 1775767657]  req@ffff9f14ab4fce00 x1862022114747776/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767682 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767657.483273:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fce00 x1862022114747776/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767682 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767657.483282:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767657.483284:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767657.483286:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114747776:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767657.483289:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114747904:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767657.483297:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767657.483302:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767657/real 1775767657]  req@ffff9f14ab4fdc00 x1862022114747904/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767682 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767657.483310:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdc00 x1862022114747904/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767682 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767657.483317:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767657.483320:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767657.483323:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114747904:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767657.483329:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114748032:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767657.483348:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767657.483359:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767657/real 1775767657]  req@ffff9f14ab4ff100 x1862022114748032/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767682 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767657.483371:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff100 x1862022114748032/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767682 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767657.483379:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767657.483382:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767657.483384:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114748032:192.168.202.110@tcp:8:kworker.0
00010000:00010000:3.0:1775767657.738723:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767658.506637:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:0.0:1775767658.762718:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767659.530675:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767659.786711:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767660.554654:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:0.0:1775767660.810695:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767661.578704:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767661.834724:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1775767662.602631:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767662.602639:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1775767662.602643:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767662.602652:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5040
00000100:00100000:2.0:1775767662.602655:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:1.0:1775767662.602662:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767662.602669:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775767662.602701:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fca80 x1862022114748160/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767662.602723:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767662.602726:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775767662.602727:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767662.602731:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5040
00000100:00080000:1.0:1775767662.602733:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767662.602736:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775767662.602742:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114748160:192.168.202.110@tcp:250:kworker.0
00000008:00100000:1.0:1775767662.602744:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767662.602760:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff800 x1862022114748288/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767662.602768:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767662.602771:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775767662.602772:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767662.602776:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5040
00000100:00080000:1.0:1775767662.602779:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767662.602784:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767662.602789:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767662.602799:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fed80 x1862022114748416/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767662.602807:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767662.602810:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775767662.602811:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767662.602814:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5040
00000100:00080000:1.0:1775767662.602817:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767662.602821:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767662.602826:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775767662.602838:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ffb80 x1862022114748544/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767662.602845:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767662.602849:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1775767662.602851:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767662.602855:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5040
00000100:00080000:1.0:1775767662.602857:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767662.602859:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767662.602864:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775767662.602872:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114748288:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775767662.602875:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff100 x1862022114748672/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767662.602893:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767662.602896:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775767662.602898:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767662.602901:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5040
00000100:00080000:1.0:1775767662.602905:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767662.602908:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767662.602913:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767662.602924:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdc00 x1862022114748800/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775767662.602924:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114748416:192.168.202.110@tcp:38:kworker.0
00000100:00080000:1.0:1775767662.602940:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767662.602942:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775767662.602943:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767662.602945:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5040
00000100:00080000:1.0:1775767662.602949:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767662.602953:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767662.602956:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767662.602965:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fce00 x1862022114748928/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775767662.602967:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ffb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114748544:192.168.202.110@tcp:8:kworker.0
00000100:00080000:1.0:1775767662.602980:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767662.602983:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775767662.602985:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767662.602990:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5040
00000100:00080000:1.0:1775767662.602994:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767662.602997:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775767662.603000:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114748672:192.168.202.110@tcp:8:kworker.0
00000008:00100000:1.0:1775767662.603001:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775767662.603018:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fc380 x1862022114749056/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775767662.603026:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114748800:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767662.603044:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114748928:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767662.603065:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114749056:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775767662.858713:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767663.626676:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775767663.882739:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767664.650661:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767664.906764:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767665.674663:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000400:00000100:3.0:1775767665.674857:0:2393:0:(lib-socket.c:454:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.202.110:988
00000400:00000100:3.0:1775767665.674900:0:2393:0:(acceptor.c:111:lnet_connect_console_error()) Connection to 192.168.202.110@tcp at host 192.168.202.110 was unreachable: the network or that node may be down, or Lustre may be misconfigured.
00000800:00000100:3.0:1775767665.674918:0:2393:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:3.0:1775767665.674960:0:2393:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:3.0:1775767665.674968:0:2393:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:3.0:1775767665.674989:0:2393:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000100:00100000:2.0:1775767665.674998:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767662/real 1775767665]  req@ffff9f14ab4ff800 x1862022114748288/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767687 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000800:00000100:3.0:1775767665.675008:0:2393:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:3.0:1775767665.675024:0:2393:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:3.0:1775767665.675041:0:2393:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000100:00100000:2.0:1775767665.675042:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff800 x1862022114748288/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767687 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000800:00000100:3.0:1775767665.675056:0:2393:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000100:00080000:2.0:1775767665.675060:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767665.675069:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767665.675076:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114748288:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767665.675086:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767662/real 1775767665]  req@ffff9f14ab4fed80 x1862022114748416/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767687 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767665.675100:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fed80 x1862022114748416/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767687 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767665.675106:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767665.675110:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767665.675114:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114748416:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767665.675122:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767662/real 1775767665]  req@ffff9f14ab4ffb80 x1862022114748544/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767687 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767665.675135:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ffb80 x1862022114748544/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767687 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767665.675144:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767665.675148:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767665.675152:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ffb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114748544:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775767665.675161:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767662/real 1775767665]  req@ffff9f14ab4fca80 x1862022114748160/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767687 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767665.675175:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fca80 x1862022114748160/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767687 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767665.675190:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767665.675195:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767665.675198:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114748160:192.168.202.110@tcp:250:kworker.0
00000100:00100000:2.0:1775767665.675236:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767662/real 1775767665]  req@ffff9f14ab4fc380 x1862022114749056/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767687 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767665.675251:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fc380 x1862022114749056/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767687 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767665.675260:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767665.675265:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767665.675269:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114749056:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775767665.675277:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767662/real 1775767665]  req@ffff9f14ab4ff100 x1862022114748672/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767687 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767665.675291:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff100 x1862022114748672/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767687 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767665.675300:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767665.675304:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767665.675308:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114748672:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775767665.675316:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767662/real 1775767665]  req@ffff9f14ab4fdc00 x1862022114748800/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767687 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767665.675332:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdc00 x1862022114748800/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767687 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767665.675342:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767665.675345:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767665.675348:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114748800:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767665.675393:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767662/real 1775767665]  req@ffff9f14ab4fce00 x1862022114748928/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767687 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767665.675408:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fce00 x1862022114748928/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767687 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767665.675416:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767665.675421:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767665.675426:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114748928:192.168.202.110@tcp:38:kworker.0
00010000:00010000:0.0:1775767665.930719:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767666.698645:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:0.0:1775767666.954721:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767667.722662:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:1.0:1775767667.722696:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767667.722710:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1775767667.722714:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767667.722723:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5045
00000100:00080000:1.0:1775767667.722732:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767667.722737:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775767667.722770:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fea00 x1862022114749184/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767667.722797:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767667.722800:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775767667.722801:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767667.722805:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5045
00000100:00080000:1.0:1775767667.722807:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767667.722809:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767667.722816:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767667.722826:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fd880 x1862022114749312/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767667.722833:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767667.722835:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775767667.722837:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767667.722839:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5045
00000100:00080000:1.0:1775767667.722841:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767667.722843:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767667.722846:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767667.722851:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fe300 x1862022114749440/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767667.722858:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767667.722860:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775767667.722862:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767667.722866:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5045
00000100:00080000:1.0:1775767667.722869:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767667.722872:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767667.722875:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775767667.722882:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdf80 x1862022114749568/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767667.722889:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767667.722892:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1775767667.722895:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:2.0:1775767667.722899:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114749184:192.168.202.110@tcp:250:kworker.0
00000100:00080000:1.0:1775767667.722906:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5045
00000100:00080000:1.0:1775767667.722914:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767667.722918:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767667.722924:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775767667.722940:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fce00 x1862022114749696/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767667.722953:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767667.722957:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775767667.722958:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767667.722962:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5045
00000100:00080000:1.0:1775767667.722965:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767667.722969:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767667.722975:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767667.722984:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdc00 x1862022114749824/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767667.722994:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767667.722998:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775767667.722999:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767667.723003:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5045
00000100:00080000:1.0:1775767667.723007:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767667.723019:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000800:00000100:2.0:1775767667.723019:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000008:00100000:1.0:1775767667.723032:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767667.723048:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff100 x1862022114749952/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767667.723059:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767667.723064:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775767667.723067:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767667.723088:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5045
00000100:00100000:2.0:1775767667.723094:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767667/real 1775767667]  req@ffff9f14ab4fea00 x1862022114749184/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767692 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767667.723098:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767667.723112:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767667.723129:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775767667.723145:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fea00 x1862022114749184/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767692 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767667.723151:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fc380 x1862022114750080/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767667.723194:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767667.723204:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767667.723212:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114749184:192.168.202.110@tcp:250:kworker.0
00000100:00100000:2.0:1775767667.723223:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114749312:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775767667.723258:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767667.723273:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767667/real 1775767667]  req@ffff9f14ab4fd880 x1862022114749312/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767692 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767667.723287:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fd880 x1862022114749312/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767692 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767667.723296:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767667.723300:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767667.723303:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114749312:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767667.723311:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114749440:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775767667.723370:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767667.723382:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767667/real 1775767667]  req@ffff9f14ab4fe300 x1862022114749440/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767692 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767667.723395:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fe300 x1862022114749440/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767692 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767667.723403:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767667.723407:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767667.723411:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114749440:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767667.723450:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114749568:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775767667.723473:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767667.723485:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767667/real 1775767667]  req@ffff9f14ab4fdf80 x1862022114749568/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767692 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767667.723498:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdf80 x1862022114749568/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767692 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767667.723506:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767667.723510:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767667.723512:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114749568:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775767667.723518:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114749696:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775767667.723537:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767667.723546:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767667/real 1775767667]  req@ffff9f14ab4fce00 x1862022114749696/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767692 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767667.723592:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fce00 x1862022114749696/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767692 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767667.723601:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767667.723605:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767667.723608:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114749696:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775767667.723614:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114749824:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775767667.723638:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767667.723649:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767667/real 1775767667]  req@ffff9f14ab4fdc00 x1862022114749824/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767692 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767667.723668:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdc00 x1862022114749824/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767692 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767667.723677:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767667.723680:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767667.723683:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114749824:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767667.723690:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114749952:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775767667.723711:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767667.723723:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767667/real 1775767667]  req@ffff9f14ab4ff100 x1862022114749952/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767692 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767667.723736:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff100 x1862022114749952/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767692 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767667.723745:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767667.723749:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767667.723752:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114749952:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767667.723759:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114750080:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775767667.723781:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767667.723793:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767667/real 1775767667]  req@ffff9f14ab4fc380 x1862022114750080/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767692 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767667.723806:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fc380 x1862022114750080/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767692 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767667.723821:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767667.723825:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767667.723828:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114750080:192.168.202.110@tcp:8:kworker.0
00010000:00010000:0.0:1775767667.978720:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767668.746697:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767669.002787:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767669.770706:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767670.026802:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767670.794685:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775767671.050824:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767671.818630:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767672.074778:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767672.842648:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:0.0:1775767672.842689:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767672.842695:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1775767672.842698:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767672.842707:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5050
00000100:00080000:0.0:1775767672.842716:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767672.842722:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775767672.842755:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d1180 x1862022114750208/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767672.842778:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767672.842780:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775767672.842782:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767672.842785:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5050
00000100:00080000:0.0:1775767672.842787:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767672.842790:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767672.842798:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767672.842809:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3480 x1862022114750336/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775767672.842810:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114750208:192.168.202.110@tcp:250:kworker.0
00000100:00080000:0.0:1775767672.842823:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767672.842828:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:0.0:1775767672.842829:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767672.842832:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5050
00000100:00080000:0.0:1775767672.842834:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767672.842836:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767672.842839:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767672.842847:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0380 x1862022114750464/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767672.842853:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767672.842855:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775767672.842856:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767672.842859:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5050
00000100:00080000:0.0:1775767672.842861:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767672.842864:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767672.842867:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775767672.842873:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d1f80 x1862022114750592/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767672.842878:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767672.842880:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1775767672.842883:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000800:00000100:2.0:1775767672.842883:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:0.0:1775767672.842889:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5050
00000100:00080000:0.0:1775767672.842892:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767672.842896:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767672.842900:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775767672.842907:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3b80 x1862022114750720/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767672.842913:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767672.842916:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775767672.842918:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767672.842921:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5050
00000100:00100000:2.0:1775767672.842925:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767672/real 1775767672]  req@ffff9f14a04d1180 x1862022114750208/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767697 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767672.842926:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767672.842933:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767672.842940:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775767672.842949:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1180 x1862022114750208/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767697 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767672.842952:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3100 x1862022114750848/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767672.842973:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767672.842974:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767672.842980:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775767672.842982:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:0.0:1775767672.842983:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:2.0:1775767672.842989:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114750208:192.168.202.110@tcp:250:kworker.0
00000100:00080000:0.0:1775767672.842990:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5050
00000100:00080000:0.0:1775767672.842996:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767672.842999:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767672.843002:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767672.843009:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0e00 x1862022114750976/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775767672.843016:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114750336:192.168.202.110@tcp:38:kworker.0
00000100:00080000:0.0:1775767672.843017:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767672.843023:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775767672.843024:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767672.843028:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5050
00000100:00080000:0.0:1775767672.843031:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767672.843034:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767672.843038:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:2.0:1775767672.843040:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767672.843050:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0000 x1862022114751104/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775767672.843054:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767672/real 1775767672]  req@ffff9f14a04d3480 x1862022114750336/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767697 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767672.843082:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3480 x1862022114750336/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767697 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767672.843087:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767672.843091:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767672.843093:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114750336:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767672.843098:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114750464:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775767672.843109:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767672.843114:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767672/real 1775767672]  req@ffff9f14a04d0380 x1862022114750464/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767697 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767672.843122:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0380 x1862022114750464/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767697 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767672.843126:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767672.843128:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767672.843130:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114750464:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767672.843134:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114750592:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775767672.843143:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767672.843148:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767672/real 1775767672]  req@ffff9f14a04d1f80 x1862022114750592/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767697 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767672.843155:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1f80 x1862022114750592/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767697 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767672.843164:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767672.843166:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767672.843168:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114750592:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775767672.843171:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114750720:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775767672.843180:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767672.843185:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767672/real 1775767672]  req@ffff9f14a04d3b80 x1862022114750720/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767697 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767672.843191:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3b80 x1862022114750720/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767697 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767672.843196:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767672.843198:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767672.843199:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114750720:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775767672.843203:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114750848:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775767672.843211:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767672.843216:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767672/real 1775767672]  req@ffff9f14a04d3100 x1862022114750848/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767697 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767672.843223:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3100 x1862022114750848/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767697 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767672.843228:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767672.843230:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767672.843232:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114750848:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767672.843244:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114750976:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775767672.843254:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767672.843260:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767672/real 1775767672]  req@ffff9f14a04d0e00 x1862022114750976/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767697 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767672.843267:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0e00 x1862022114750976/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767697 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767672.843271:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767672.843273:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767672.843275:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114750976:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767672.843278:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114751104:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775767672.843305:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767672.843313:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767672/real 1775767672]  req@ffff9f14a04d0000 x1862022114751104/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767697 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767672.843320:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0000 x1862022114751104/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767697 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767672.843325:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767672.843327:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767672.843329:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114751104:192.168.202.110@tcp:8:kworker.0
00010000:00010000:0.0:1775767673.098735:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767673.866630:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775767674.122731:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767674.890654:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:0.0:1775767675.146698:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767675.914666:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767676.170704:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767676.938666:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767677.194701:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1775767677.962669:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:2.0:1775767677.962670:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:0.0:1775767677.962679:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1775767677.962683:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767677.962690:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5055
00000100:00080000:0.0:1775767677.962700:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767677.962705:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775767677.962740:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d1500 x1862022114751232/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767677.962763:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767677.962766:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775767677.962767:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767677.962770:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5055
00000100:00080000:0.0:1775767677.962772:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767677.962774:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767677.962782:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767677.962791:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3800 x1862022114751360/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767677.962799:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767677.962801:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:0.0:1775767677.962802:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:2.0:1775767677.962806:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114751232:192.168.202.110@tcp:250:kworker.0
00000100:00080000:0.0:1775767677.962808:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5055
00000100:00080000:0.0:1775767677.962812:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767677.962817:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767677.962820:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767677.962828:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d2a00 x1862022114751488/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767677.962834:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767677.962836:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775767677.962837:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767677.962840:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5055
00000100:00080000:0.0:1775767677.962842:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767677.962845:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767677.962848:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775767677.962855:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d2300 x1862022114751616/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767677.962859:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767677.962862:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1775767677.962863:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767677.962866:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5055
00000100:00080000:0.0:1775767677.962869:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767677.962871:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767677.962874:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:2.0:1775767677.962881:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767677.962884:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0a80 x1862022114751744/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767677.962891:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767677.962893:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775767677.962895:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767677.962898:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5055
00000100:00080000:0.0:1775767677.962903:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767677.962906:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767677.962909:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767677.962917:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0000 x1862022114751872/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767677.962923:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767677.962926:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:0.0:1775767677.962929:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:2.0:1775767677.962929:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767677/real 1775767677]  req@ffff9f14a04d1500 x1862022114751232/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767702 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767677.962933:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5055
00000100:00080000:0.0:1775767677.962938:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767677.962943:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767677.962949:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767677.962958:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0e00 x1862022114752000/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775767677.962963:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1500 x1862022114751232/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767702 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767677.962967:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767677.962975:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775767677.962978:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767677.962985:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5055
00000100:00080000:0.0:1775767677.962988:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767677.962989:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767677.962991:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767677.962997:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00080000:2.0:1775767677.963000:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767677.963007:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3100 x1862022114752128/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775767677.963008:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114751232:192.168.202.110@tcp:250:kworker.0
00000100:00100000:2.0:1775767677.963040:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114751360:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775767677.963063:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767677.963074:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767677/real 1775767677]  req@ffff9f14a04d3800 x1862022114751360/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767702 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767677.963086:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3800 x1862022114751360/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767702 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767677.963095:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767677.963098:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767677.963101:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114751360:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767677.963110:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114751488:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775767677.963128:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767677.963137:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767677/real 1775767677]  req@ffff9f14a04d2a00 x1862022114751488/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767702 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767677.963147:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d2a00 x1862022114751488/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767702 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767677.963155:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767677.963158:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767677.963161:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114751488:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767677.963166:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114751616:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775767677.963182:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767677.963190:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767677/real 1775767677]  req@ffff9f14a04d2300 x1862022114751616/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767702 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767677.963201:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d2300 x1862022114751616/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767702 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767677.963209:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767677.963212:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767677.963215:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114751616:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775767677.963220:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114751744:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775767677.963251:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767677.963264:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767677/real 1775767677]  req@ffff9f14a04d0a80 x1862022114751744/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767702 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767677.963276:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0a80 x1862022114751744/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767702 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767677.963284:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767677.963286:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767677.963289:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114751744:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775767677.963295:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114751872:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775767677.963313:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767677.963323:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767677/real 1775767677]  req@ffff9f14a04d0000 x1862022114751872/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767702 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767677.963334:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0000 x1862022114751872/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767702 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767677.963343:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767677.963346:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767677.963348:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114751872:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767677.963354:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114752000:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775767677.963370:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767677.963380:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767677/real 1775767677]  req@ffff9f14a04d0e00 x1862022114752000/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767702 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767677.963391:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0e00 x1862022114752000/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767702 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767677.963404:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767677.963408:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767677.963411:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114752000:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767677.963416:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114752128:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775767677.963435:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767677.963445:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767677/real 1775767677]  req@ffff9f14a04d3100 x1862022114752128/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767702 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767677.963456:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3100 x1862022114752128/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767702 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767677.963465:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767677.963468:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767677.963472:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114752128:192.168.202.110@tcp:8:kworker.0
00010000:00010000:1.0:1775767678.218735:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767678.986664:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767679.242739:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767680.010680:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:0.0:1775767680.266681:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767681.034697:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775767681.290693:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767682.058648:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767682.314683:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767683.082657:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775767683.082682:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767683.082690:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775767683.082694:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767683.082704:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5060
00000100:00080000:3.0:1775767683.082715:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767683.082721:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775767683.082779:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744700 x1862022114752256/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767683.082812:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767683.082816:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775767683.082819:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767683.082825:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5060
00000100:00080000:3.0:1775767683.082831:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767683.082836:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775767683.082862:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114752256:192.168.202.110@tcp:250:kworker.0
00000008:00100000:3.0:1775767683.082865:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767683.082881:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745f80 x1862022114752384/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767683.082892:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767683.082896:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775767683.082898:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767683.082902:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5060
00000100:00080000:3.0:1775767683.082917:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767683.082922:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767683.082927:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000800:00000100:2.0:1775767683.082934:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767683.082940:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747800 x1862022114752512/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767683.082952:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767683.082956:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775767683.082958:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767683.082962:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5060
00000100:00080000:3.0:1775767683.082966:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767683.082970:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767683.082976:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775767683.082979:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767683/real 1775767683]  req@ffff9f1485744700 x1862022114752256/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767708 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767683.082990:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744000 x1862022114752640/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775767683.082997:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744700 x1862022114752256/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767708 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767683.083007:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767683.083011:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767683.083012:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775767683.083017:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767683.083018:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:3.0:1775767683.083028:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5060
00000100:00100000:2.0:1775767683.083029:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114752256:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775767683.083037:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767683.083043:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767683.083048:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775767683.083053:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114752384:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767683.083062:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747100 x1862022114752768/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:2.0:1775767683.083075:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775767683.083076:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767683.083080:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775767683.083083:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:2.0:1775767683.083086:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767683/real 1775767683]  req@ffff9f1485745f80 x1862022114752384/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767708 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767683.083091:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5060
00000100:00080000:3.0:1775767683.083101:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:2.0:1775767683.083104:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745f80 x1862022114752384/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767708 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767683.083109:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00080000:2.0:1775767683.083116:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000008:00100000:3.0:1775767683.083119:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00080000:2.0:1775767683.083127:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767683.083131:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114752384:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767683.083135:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745180 x1862022114752896/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775767683.083139:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114752512:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775767683.083155:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000800:00000100:2.0:1775767683.083161:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775767683.083162:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775767683.083165:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:2.0:1775767683.083172:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767683/real 1775767683]  req@ffff9f1485747800 x1862022114752512/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767708 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767683.083172:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5060
00000100:00080000:3.0:1775767683.083180:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:2.0:1775767683.083206:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747800 x1862022114752512/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767708 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767683.083206:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767683.083216:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00080000:2.0:1775767683.083222:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767683.083228:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767683.083232:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745c00 x1862022114753024/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775767683.083235:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114752512:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767683.083249:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114752640:192.168.202.110@tcp:8:kworker.0
00000100:00080000:3.0:1775767683.083250:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767683.083256:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775767683.083257:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767683.083261:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5060
00000100:00080000:3.0:1775767683.083265:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767683.083269:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767683.083273:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:2.0:1775767683.083276:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767683.083288:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767683/real 1775767683]  req@ffff9f1485744000 x1862022114752640/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767708 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767683.083292:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746680 x1862022114753152/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775767683.083302:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744000 x1862022114752640/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767708 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767683.083325:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767683.083330:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767683.083334:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114752640:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775767683.083350:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114752768:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775767683.083361:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767683.083367:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767683/real 1775767683]  req@ffff9f1485747100 x1862022114752768/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767708 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767683.083374:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747100 x1862022114752768/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767708 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767683.083379:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767683.083382:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767683.083383:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114752768:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775767683.083387:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114752896:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775767683.083396:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767683.083401:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767683/real 1775767683]  req@ffff9f1485745180 x1862022114752896/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767708 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767683.083409:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745180 x1862022114752896/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767708 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767683.083414:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767683.083415:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767683.083417:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114752896:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767683.083420:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114753024:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775767683.083429:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767683.083434:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767683/real 1775767683]  req@ffff9f1485745c00 x1862022114753024/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767708 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767683.083442:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745c00 x1862022114753024/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767708 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767683.083446:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767683.083448:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767683.083450:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114753024:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767683.083453:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114753152:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775767683.083462:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767683.083466:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767683/real 1775767683]  req@ffff9f1485746680 x1862022114753152/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767708 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767683.083473:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746680 x1862022114753152/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767708 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767683.083481:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767683.083484:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767683.083486:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114753152:192.168.202.110@tcp:8:kworker.0
00010000:00010000:0.0:1775767683.338730:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767684.106662:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767684.362701:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767685.130647:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775767685.386716:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767686.154655:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767686.410684:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767687.178715:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:0.0:1775767687.434688:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767688.202634:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:2.0:1775767688.202647:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767688.202653:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1775767688.202656:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767688.202666:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5065
00000100:00080000:2.0:1775767688.202676:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767688.202684:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775767688.202720:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1c00 x1862022114753280/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767688.202762:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767688.202766:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775767688.202768:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767688.202772:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5065
00000100:00080000:2.0:1775767688.202776:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767688.202781:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767688.202791:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775767688.202805:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3480 x1862022114753408/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767688.202816:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767688.202820:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775767688.202822:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767688.202826:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5065
00000100:00080000:2.0:1775767688.202830:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767688.202834:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767688.202838:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767688.202842:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114753280:192.168.202.110@tcp:250:kworker.0
00000100:00100000:2.0:1775767688.202849:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2300 x1862022114753536/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767688.202858:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767688.202861:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775767688.202863:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767688.202866:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5065
00000100:00080000:2.0:1775767688.202869:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767688.202872:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767688.202877:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775767688.202885:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3100 x1862022114753664/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767688.202893:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767688.202896:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000800:00000100:0.0:1775767688.202897:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:2.0:1775767688.202898:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767688.202901:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5065
00000100:00080000:2.0:1775767688.202904:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767688.202907:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767688.202912:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775767688.202920:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1880 x1862022114753792/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767688.202929:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1775767688.202932:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767688/real 1775767688]  req@ffff9f14852d1c00 x1862022114753280/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767713 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767688.202933:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775767688.202935:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767688.202938:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5065
00000100:00080000:2.0:1775767688.202941:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767688.202945:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775767688.202948:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1c00 x1862022114753280/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767713 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:2.0:1775767688.202949:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775767688.202957:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0e00 x1862022114753920/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767688.202959:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767688.202963:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:2.0:1775767688.202966:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1775767688.202967:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114753280:192.168.202.110@tcp:250:kworker.0
00000100:00080000:2.0:1775767688.202970:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775767688.202971:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775767688.202974:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114753408:192.168.202.110@tcp:38:kworker.0
00000100:00080000:2.0:1775767688.202975:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5065
00000100:00080000:2.0:1775767688.202978:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767688.202982:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767688.202986:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000800:00000100:0.0:1775767688.202994:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767688.202995:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3b80 x1862022114754048/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767688.203004:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1775767688.203005:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767688/real 1775767688]  req@ffff9f14852d3480 x1862022114753408/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767713 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767688.203007:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775767688.203010:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767688.203014:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5065
00000100:00100000:0.0:1775767688.203017:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3480 x1862022114753408/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767713 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767688.203017:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767688.203020:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767688.203025:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00080000:0.0:1775767688.203026:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767688.203029:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767688.203033:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114753408:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767688.203034:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0000 x1862022114754176/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775767688.203103:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114753536:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767688.203120:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767688.203129:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767688/real 1775767688]  req@ffff9f14852d2300 x1862022114753536/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767713 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767688.203155:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2300 x1862022114753536/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767713 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767688.203161:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767688.203163:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767688.203165:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114753536:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767688.203169:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114753664:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767688.203182:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767688.203189:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767688/real 1775767688]  req@ffff9f14852d3100 x1862022114753664/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767713 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767688.203196:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3100 x1862022114753664/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767713 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767688.203201:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767688.203203:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767688.203204:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114753664:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775767688.203208:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114753792:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767688.203217:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767688.203222:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767688/real 1775767688]  req@ffff9f14852d1880 x1862022114753792/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767713 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767688.203229:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1880 x1862022114753792/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767713 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767688.203234:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767688.203236:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767688.203237:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114753792:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775767688.203240:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114753920:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767688.203250:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767688.203255:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767688/real 1775767688]  req@ffff9f14852d0e00 x1862022114753920/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767713 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767688.203267:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0e00 x1862022114753920/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767713 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767688.203272:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767688.203273:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767688.203275:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114753920:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767688.203278:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114754048:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767688.203287:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767688.203292:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767688/real 1775767688]  req@ffff9f14852d3b80 x1862022114754048/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767713 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767688.203298:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3b80 x1862022114754048/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767713 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767688.203303:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767688.203305:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767688.203307:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114754048:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767688.203310:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114754176:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767688.203319:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767688.203323:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767688/real 1775767688]  req@ffff9f14852d0000 x1862022114754176/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767713 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767688.203330:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0000 x1862022114754176/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767713 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767688.203339:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767688.203341:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767688.203343:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114754176:192.168.202.110@tcp:8:kworker.0
00010000:00010000:1.0:1775767688.458696:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767689.226678:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767689.484081:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767690.250682:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:0.0:1775767690.506768:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767691.274639:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767691.530736:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767692.298654:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775767692.554732:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767693.322712:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:2.0:1775767693.322726:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767693.322734:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1775767693.322739:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767693.322750:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5070
00000100:00080000:2.0:1775767693.322763:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767693.322897:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775767693.322973:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0380 x1862022114754304/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767693.322997:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767693.323000:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775767693.323003:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767693.323007:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5070
00000100:00080000:2.0:1775767693.323016:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767693.323020:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767693.323029:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775767693.323038:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1500 x1862022114754432/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767693.323061:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767693.323064:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775767693.323065:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767693.323067:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5070
00000100:00080000:2.0:1775767693.323069:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767693.323072:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767693.323075:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775767693.323080:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2a00 x1862022114754560/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767693.323102:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767693.323106:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775767693.323108:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767693.323111:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5070
00000100:00080000:2.0:1775767693.323114:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767693.323117:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767693.323122:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775767693.323129:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114754304:192.168.202.110@tcp:250:kworker.0
00000100:00100000:2.0:1775767693.323133:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1180 x1862022114754688/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767693.323143:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767693.323145:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1775767693.323146:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767693.323149:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5070
00000100:00080000:2.0:1775767693.323151:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767693.323154:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767693.323157:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775767693.323166:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2d80 x1862022114754816/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767693.323172:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767693.323174:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775767693.323175:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767693.323179:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5070
00000100:00080000:2.0:1775767693.323182:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767693.323184:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767693.323188:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000800:00000100:0.0:1775767693.323192:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767693.323199:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3800 x1862022114754944/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767693.323206:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767693.323208:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775767693.323209:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767693.323212:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5070
00000100:00080000:2.0:1775767693.323215:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767693.323218:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767693.323222:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775767693.323229:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0a80 x1862022114755072/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775767693.323240:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767693/real 1775767693]  req@ffff9f14852d0380 x1862022114754304/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767718 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767693.323242:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767693.323247:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775767693.323250:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767693.323257:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5070
00000100:00080000:2.0:1775767693.323262:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767693.323267:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767693.323271:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775767693.323281:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0380 x1862022114754304/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767718 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767693.323281:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0700 x1862022114755200/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767693.323311:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767693.323316:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767693.323321:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114754304:192.168.202.110@tcp:250:kworker.0
00000100:00100000:0.0:1775767693.323328:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114754432:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767693.323349:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767693.323360:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767693/real 1775767693]  req@ffff9f14852d1500 x1862022114754432/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767718 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767693.323373:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1500 x1862022114754432/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767718 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767693.323379:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767693.323381:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767693.323383:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114754432:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767693.323387:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114754560:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767693.323397:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767693.323403:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767693/real 1775767693]  req@ffff9f14852d2a00 x1862022114754560/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767718 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767693.323411:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2a00 x1862022114754560/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767718 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767693.323429:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767693.323433:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767693.323436:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114754560:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767693.323468:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114754688:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767693.323489:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767693.323498:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767693/real 1775767693]  req@ffff9f14852d1180 x1862022114754688/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767718 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767693.323506:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1180 x1862022114754688/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767718 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767693.323516:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767693.323519:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767693.323521:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114754688:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775767693.323525:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114754816:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767693.323535:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767693.323541:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767693/real 1775767693]  req@ffff9f14852d2d80 x1862022114754816/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767718 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767693.323548:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2d80 x1862022114754816/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767718 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767693.323573:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767693.323575:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767693.323577:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114754816:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775767693.323581:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114754944:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767693.323594:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767693.323601:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767693/real 1775767693]  req@ffff9f14852d3800 x1862022114754944/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767718 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767693.323609:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3800 x1862022114754944/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767718 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767693.323614:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767693.323616:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767693.323619:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114754944:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767693.323622:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114755072:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767693.323632:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767693.323638:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767693/real 1775767693]  req@ffff9f14852d0a80 x1862022114755072/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767718 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767693.323645:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0a80 x1862022114755072/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767718 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767693.323650:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767693.323652:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767693.323653:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114755072:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767693.323657:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114755200:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767693.323666:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767693.323671:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767693/real 1775767693]  req@ffff9f14852d0700 x1862022114755200/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767718 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767693.323678:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0700 x1862022114755200/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767718 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767693.323683:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767693.323685:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767693.323687:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114755200:192.168.202.110@tcp:8:kworker.0
00010000:00010000:3.0:1775767693.578692:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767694.346658:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:0.0:1775767694.602705:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767695.370646:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775767695.626682:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767696.394660:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767696.650661:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767697.418650:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:0.0:1775767697.674663:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767698.442669:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:1.0:1775767698.442669:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767698.442678:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1775767698.442682:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767698.442692:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5075
00000100:00080000:1.0:1775767698.442702:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767698.442708:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775767698.442745:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fca80 x1862022114755328/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767698.442776:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767698.442781:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775767698.442783:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767698.442789:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5075
00000100:00080000:1.0:1775767698.442794:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767698.442799:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767698.442810:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775767698.442810:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114755328:192.168.202.110@tcp:250:kworker.0
00000100:00100000:1.0:1775767698.442832:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ffb80 x1862022114755456/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767698.442844:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767698.442846:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775767698.442848:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767698.442853:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5075
00000100:00080000:1.0:1775767698.442856:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767698.442860:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767698.442864:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000800:00000100:2.0:1775767698.442874:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767698.442876:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fed80 x1862022114755584/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767698.442884:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767698.442886:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775767698.442888:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767698.442891:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5075
00000100:00080000:1.0:1775767698.442893:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767698.442895:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767698.442898:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775767698.442909:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff800 x1862022114755712/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775767698.442910:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767698/real 1775767698]  req@ffff9f14ab4fca80 x1862022114755328/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767723 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767698.442950:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767698.442954:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00100000:2.0:1775767698.442955:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fca80 x1862022114755328/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767723 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767698.442958:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767698.442969:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5075
00000100:00080000:2.0:1775767698.442972:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767698.442974:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767698.442980:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:1.0:1775767698.442981:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767698.442989:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775767698.442990:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114755328:192.168.202.110@tcp:250:kworker.0
00000100:00100000:1.0:1775767698.443003:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fc380 x1862022114755840/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767698.443010:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767698.443013:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775767698.443014:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767698.443017:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5075
00000100:00080000:1.0:1775767698.443020:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:2.0:1775767698.443022:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ffb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114755456:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775767698.443074:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:1.0:1775767698.443083:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767698.443088:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775767698.443091:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767698/real 1775767698]  req@ffff9f14ab4ffb80 x1862022114755456/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767723 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767698.443100:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff100 x1862022114755968/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767698.443115:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:2.0:1775767698.443117:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ffb80 x1862022114755456/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767723 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767698.443119:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775767698.443122:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767698.443130:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5075
00000100:00080000:2.0:1775767698.443133:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767698.443135:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767698.443138:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00080000:1.0:1775767698.443141:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775767698.443145:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ffb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114755456:192.168.202.110@tcp:38:kworker.0
00000008:00100000:1.0:1775767698.443147:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775767698.443152:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114755584:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775767698.443158:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdc00 x1862022114756096/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767698.443166:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767698.443169:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775767698.443170:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767698.443176:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5075
00000800:00000100:2.0:1775767698.443176:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:1.0:1775767698.443181:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767698.443185:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767698.443190:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775767698.443193:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767698/real 1775767698]  req@ffff9f14ab4fed80 x1862022114755584/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767723 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767698.443203:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fce00 x1862022114756224/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775767698.443206:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fed80 x1862022114755584/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767723 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767698.443236:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767698.443239:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767698.443243:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114755584:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767698.443249:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114755712:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775767698.443264:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767698.443271:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767698/real 1775767698]  req@ffff9f14ab4ff800 x1862022114755712/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767723 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767698.443279:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff800 x1862022114755712/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767723 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767698.443290:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767698.443293:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767698.443296:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114755712:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775767698.443301:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114755840:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775767698.443313:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767698.443319:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767698/real 1775767698]  req@ffff9f14ab4fc380 x1862022114755840/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767723 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767698.443326:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fc380 x1862022114755840/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767723 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767698.443332:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767698.443333:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767698.443335:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114755840:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775767698.443347:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114755968:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775767698.443358:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767698.443363:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767698/real 1775767698]  req@ffff9f14ab4ff100 x1862022114755968/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767723 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767698.443371:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff100 x1862022114755968/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767723 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767698.443375:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767698.443377:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767698.443379:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114755968:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767698.443383:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114756096:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775767698.443393:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767698.443397:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767698/real 1775767698]  req@ffff9f14ab4fdc00 x1862022114756096/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767723 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767698.443404:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdc00 x1862022114756096/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767723 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767698.443409:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767698.443411:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767698.443413:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114756096:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767698.443416:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114756224:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775767698.443424:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767698.443429:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767698/real 1775767698]  req@ffff9f14ab4fce00 x1862022114756224/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767723 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767698.443435:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fce00 x1862022114756224/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767723 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767698.443440:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767698.443442:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767698.443443:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114756224:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775767698.698699:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767699.466622:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767699.722755:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767700.490605:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767700.746806:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767701.514642:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:0.0:1775767701.770686:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767702.538649:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775767702.794738:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767703.562709:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775767703.562709:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767703.562721:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775767703.562725:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767703.562735:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5081
00000100:00080000:3.0:1775767703.562747:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767703.562755:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775767703.562798:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745500 x1862022114756352/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767703.562825:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767703.562828:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775767703.562830:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767703.562833:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5081
00000100:00080000:3.0:1775767703.562836:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767703.562839:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767703.562849:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775767703.562857:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114756352:192.168.202.110@tcp:250:kworker.0
00000100:00100000:3.0:1775767703.562860:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744e00 x1862022114756480/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767703.562870:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767703.562873:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775767703.562875:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767703.562877:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5081
00000100:00080000:3.0:1775767703.562879:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767703.562881:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767703.562884:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767703.562891:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744380 x1862022114756608/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767703.562898:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767703.562900:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775767703.562901:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767703.562903:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5081
00000100:00080000:3.0:1775767703.562919:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767703.562922:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767703.562926:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775767703.562933:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746a00 x1862022114756736/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:2.0:1775767703.562940:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775767703.562946:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767703.562949:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775767703.562950:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767703.562954:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5081
00000100:00080000:3.0:1775767703.562956:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767703.562958:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767703.562961:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775767703.562969:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744a80 x1862022114756864/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767703.562974:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767703.562977:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775767703.562978:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767703.562981:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5081
00000100:00080000:3.0:1775767703.562983:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767703.563004:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767703.563007:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767703.563016:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747480 x1862022114756992/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767703.563022:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767703.563025:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775767703.563027:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767703.563032:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5081
00000100:00100000:2.0:1775767703.563035:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767703/real 1775767703]  req@ffff9f1485745500 x1862022114756352/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767728 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767703.563036:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767703.563041:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767703.563047:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767703.563059:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745880 x1862022114757120/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767703.563066:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:2.0:1775767703.563069:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745500 x1862022114756352/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767728 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767703.563070:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775767703.563074:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767703.563083:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5081
00000100:00080000:3.0:1775767703.563087:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767703.563089:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767703.563089:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767703.563096:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00080000:2.0:1775767703.563098:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767703.563104:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114756352:192.168.202.110@tcp:250:kworker.0
00000100:00100000:3.0:1775767703.563107:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746300 x1862022114757248/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775767703.563152:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114756480:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775767703.563179:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767703.563191:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767703/real 1775767703]  req@ffff9f1485744e00 x1862022114756480/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767728 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767703.563201:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744e00 x1862022114756480/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767728 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767703.563207:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767703.563209:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767703.563211:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114756480:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767703.563216:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114756608:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775767703.563228:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767703.563234:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767703/real 1775767703]  req@ffff9f1485744380 x1862022114756608/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767728 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767703.563246:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744380 x1862022114756608/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767728 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767703.563251:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767703.563253:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767703.563255:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114756608:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767703.563259:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114756736:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775767703.563270:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767703.563275:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767703/real 1775767703]  req@ffff9f1485746a00 x1862022114756736/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767728 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767703.563283:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746a00 x1862022114756736/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767728 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767703.563288:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767703.563290:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767703.563291:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114756736:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775767703.563295:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114756864:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775767703.563304:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767703.563309:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767703/real 1775767703]  req@ffff9f1485744a80 x1862022114756864/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767728 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767703.563317:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744a80 x1862022114756864/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767728 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767703.563326:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767703.563329:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767703.563331:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114756864:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775767703.563334:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114756992:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775767703.563344:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767703.563349:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767703/real 1775767703]  req@ffff9f1485747480 x1862022114756992/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767728 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767703.563355:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747480 x1862022114756992/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767728 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767703.563361:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767703.563363:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767703.563365:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114756992:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767703.563368:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114757120:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775767703.563377:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767703.563382:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767703/real 1775767703]  req@ffff9f1485745880 x1862022114757120/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767728 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767703.563388:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745880 x1862022114757120/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767728 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767703.563393:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767703.563395:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767703.563398:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114757120:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767703.563402:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114757248:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775767703.563411:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767703.563416:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767703/real 1775767703]  req@ffff9f1485746300 x1862022114757248/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767728 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767703.563423:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746300 x1862022114757248/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767728 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767703.563429:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767703.563430:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767703.563432:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114757248:192.168.202.110@tcp:8:kworker.0
00010000:00010000:3.0:1775767703.818730:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767704.586679:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:0.0:1775767704.842736:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767705.610705:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767705.866722:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767706.634668:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767706.890769:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767707.658750:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767707.914740:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1775767708.682642:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1775767708.682646:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:2.0:1775767708.682651:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1775767708.682656:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767708.682665:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5086
00000100:00080000:2.0:1775767708.682675:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767708.682681:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775767708.682719:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2680 x1862022114757376/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767708.682742:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767708.682745:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775767708.682746:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767708.682750:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5086
00000100:00080000:2.0:1775767708.682752:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767708.682754:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767708.682764:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775767708.682772:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1f80 x1862022114757504/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767708.682781:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1775767708.682783:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114757376:192.168.202.110@tcp:250:kworker.0
00000100:00080000:2.0:1775767708.682784:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775767708.682794:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767708.682798:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5086
00000100:00080000:2.0:1775767708.682800:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767708.682803:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767708.682806:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775767708.682815:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0700 x1862022114757632/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767708.682821:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767708.682824:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775767708.682825:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767708.682827:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5086
00000100:00080000:2.0:1775767708.682830:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767708.682832:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767708.682835:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775767708.682842:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0a80 x1862022114757760/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767708.682851:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000800:00000100:0.0:1775767708.682852:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:2.0:1775767708.682855:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1775767708.682858:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767708.682861:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5086
00000100:00080000:2.0:1775767708.682864:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767708.682866:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767708.682870:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775767708.682875:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3800 x1862022114757888/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767708.682884:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767708.682887:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775767708.682889:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767708.682893:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5086
00000100:00080000:2.0:1775767708.682898:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767708.682903:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775767708.682909:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767708/real 1775767708]  req@ffff9f14852d2680 x1862022114757376/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767733 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:2.0:1775767708.682911:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775767708.682928:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2d80 x1862022114758016/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775767708.682955:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2680 x1862022114757376/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767733 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767708.682968:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767708.682974:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775767708.682975:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767708.682979:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767708.682980:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5086
00000100:00080000:2.0:1775767708.682987:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767708.682989:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:2.0:1775767708.682994:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775767708.682998:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114757376:192.168.202.110@tcp:250:kworker.0
00000008:00100000:2.0:1775767708.683001:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775767708.683014:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1180 x1862022114758144/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767708.683021:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767708.683023:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775767708.683024:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767708.683027:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5086
00000100:00080000:2.0:1775767708.683030:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1775767708.683033:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114757504:192.168.202.110@tcp:38:kworker.0
00000100:00080000:2.0:1775767708.683035:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767708.683040:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775767708.683048:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2a00 x1862022114758272/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1775767708.683085:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767708.683100:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767708/real 1775767708]  req@ffff9f14852d1f80 x1862022114757504/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767733 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767708.683122:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1f80 x1862022114757504/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767733 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767708.683131:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767708.683135:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767708.683138:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114757504:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767708.683145:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114757632:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767708.683164:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767708.683175:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767708/real 1775767708]  req@ffff9f14852d0700 x1862022114757632/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767733 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767708.683187:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0700 x1862022114757632/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767733 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767708.683196:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767708.683200:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767708.683203:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114757632:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767708.683210:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114757760:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767708.683229:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767708.683239:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767708/real 1775767708]  req@ffff9f14852d0a80 x1862022114757760/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767733 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767708.683259:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0a80 x1862022114757760/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767733 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767708.683268:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767708.683271:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767708.683275:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114757760:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775767708.683281:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114757888:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767708.683299:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767708.683309:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767708/real 1775767708]  req@ffff9f14852d3800 x1862022114757888/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767733 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767708.683322:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3800 x1862022114757888/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767733 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767708.683330:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767708.683334:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767708.683337:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114757888:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775767708.683344:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114758016:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767708.683362:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767708.683373:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767708/real 1775767708]  req@ffff9f14852d2d80 x1862022114758016/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767733 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767708.683384:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2d80 x1862022114758016/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767733 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767708.683399:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767708.683403:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767708.683406:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114758016:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767708.683420:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114758144:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767708.683441:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767708.683451:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767708/real 1775767708]  req@ffff9f14852d1180 x1862022114758144/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767733 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767708.683463:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1180 x1862022114758144/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767733 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767708.683471:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767708.683474:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767708.683478:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114758144:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767708.683499:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114758272:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767708.683518:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767708.683529:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767708/real 1775767708]  req@ffff9f14852d2a00 x1862022114758272/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767733 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767708.683541:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2a00 x1862022114758272/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767733 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767708.683550:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767708.683553:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767708.683575:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114758272:192.168.202.110@tcp:8:kworker.0
00010000:00010000:0.0:1775767708.938680:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767709.706712:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767709.962796:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767710.730632:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767710.986682:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767711.754685:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:0.0:1775767712.010668:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767712.778646:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767713.034729:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1775767713.802663:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1775767713.802667:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:2.0:1775767713.802672:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1775767713.802677:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767713.802685:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5091
00000100:00080000:2.0:1775767713.802695:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767713.802701:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775767713.802752:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1500 x1862022114758400/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767713.802782:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767713.802787:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775767713.802788:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767713.802792:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5091
00000100:00080000:2.0:1775767713.802796:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767713.802800:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767713.802808:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767713.802814:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114758400:192.168.202.110@tcp:250:kworker.0
00000100:00100000:2.0:1775767713.802821:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0380 x1862022114758528/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767713.802837:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767713.802839:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775767713.802841:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767713.802844:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5091
00000100:00080000:2.0:1775767713.802848:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767713.802850:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767713.802853:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775767713.802861:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0000 x1862022114758656/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767713.802868:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000800:00000100:0.0:1775767713.802871:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:2.0:1775767713.802871:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775767713.802875:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767713.802879:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5091
00000100:00080000:2.0:1775767713.802881:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767713.802884:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767713.802906:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775767713.802916:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3b80 x1862022114758784/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767713.802923:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767713.802926:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1775767713.802927:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775767713.802931:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767713/real 1775767713]  req@ffff9f14852d1500 x1862022114758400/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767738 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767713.802933:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5091
00000100:00080000:2.0:1775767713.802939:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767713.802945:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767713.802950:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775767713.802954:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1500 x1862022114758400/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767738 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767713.802959:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0e00 x1862022114758912/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767713.802976:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767713.802981:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767713.802988:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114758400:192.168.202.110@tcp:250:kworker.0
00000100:00080000:2.0:1775767713.802989:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767713.802996:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775767713.802999:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767713.803002:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5091
00000100:00080000:2.0:1775767713.803005:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767713.803009:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767713.803012:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775767713.803018:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1880 x1862022114759040/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775767713.803022:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114758528:192.168.202.110@tcp:38:kworker.0
00000100:00080000:2.0:1775767713.803031:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767713.803033:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775767713.803035:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767713.803038:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5091
00000100:00080000:2.0:1775767713.803041:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767713.803043:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767713.803047:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000800:00000100:0.0:1775767713.803058:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767713.803061:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3100 x1862022114759168/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767713.803069:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767713.803071:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775767713.803073:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767713.803077:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5091
00000100:00100000:0.0:1775767713.803078:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767713/real 1775767713]  req@ffff9f14852d0380 x1862022114758528/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767738 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767713.803080:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767713.803084:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767713.803089:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775767713.803100:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2300 x1862022114759296/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775767713.803107:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0380 x1862022114758528/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767738 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767713.803118:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767713.803121:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767713.803125:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114758528:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767713.803130:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114758656:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767713.803146:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767713.803154:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767713/real 1775767713]  req@ffff9f14852d0000 x1862022114758656/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767738 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767713.803162:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0000 x1862022114758656/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767738 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767713.803167:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767713.803169:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767713.803171:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114758656:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767713.803175:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114758784:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767713.803185:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767713.803190:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767713/real 1775767713]  req@ffff9f14852d3b80 x1862022114758784/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767738 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767713.803197:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3b80 x1862022114758784/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767738 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767713.803203:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767713.803204:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767713.803206:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114758784:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775767713.803209:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114758912:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767713.803219:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767713.803224:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767713/real 1775767713]  req@ffff9f14852d0e00 x1862022114758912/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767738 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767713.803232:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0e00 x1862022114758912/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767738 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767713.803236:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767713.803238:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767713.803240:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114758912:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775767713.803251:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114759040:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767713.803261:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767713.803266:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767713/real 1775767713]  req@ffff9f14852d1880 x1862022114759040/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767738 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767713.803273:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1880 x1862022114759040/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767738 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767713.803277:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767713.803279:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767713.803281:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114759040:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767713.803284:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114759168:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767713.803293:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767713.803298:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767713/real 1775767713]  req@ffff9f14852d3100 x1862022114759168/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767738 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767713.803310:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3100 x1862022114759168/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767738 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767713.803314:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767713.803316:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767713.803318:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114759168:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767713.803321:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114759296:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767713.803329:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767713.803334:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767713/real 1775767713]  req@ffff9f14852d2300 x1862022114759296/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767738 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767713.803341:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2300 x1862022114759296/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767738 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767713.803346:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767713.803347:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767713.803349:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114759296:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775767714.058751:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767714.826712:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767715.082695:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767715.850689:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:0.0:1775767716.106752:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767716.874706:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767717.130780:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767717.898712:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767718.154765:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1775767718.922631:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767718.922639:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00100000:0.0:1775767718.922643:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:2.0:1775767718.922643:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767718.922664:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5096
00000100:00080000:2.0:1775767718.922673:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767718.922680:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775767718.922717:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3480 x1862022114759424/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767718.922742:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767718.922745:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775767718.922746:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767718.922749:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5096
00000100:00080000:2.0:1775767718.922751:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767718.922753:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767718.922761:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775767718.922768:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1c00 x1862022114759552/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767718.922775:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767718.922777:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775767718.922778:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767718.922780:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5096
00000100:00080000:2.0:1775767718.922782:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767718.922784:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767718.922786:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775767718.922793:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2300 x1862022114759680/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767718.922798:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767718.922800:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775767718.922802:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767718.922804:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5096
00000100:00080000:2.0:1775767718.922807:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767718.922810:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767718.922814:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775767718.922821:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3100 x1862022114759808/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775767718.922825:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114759424:192.168.202.110@tcp:250:kworker.0
00000100:00080000:2.0:1775767718.922837:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767718.922852:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1775767718.922853:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767718.922856:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5096
00000100:00080000:2.0:1775767718.922859:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767718.922861:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767718.922865:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775767718.922873:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1880 x1862022114759936/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767718.922879:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767718.922882:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775767718.922883:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767718.922885:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5096
00000100:00080000:2.0:1775767718.922888:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767718.922890:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767718.922893:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000800:00000100:0.0:1775767718.922899:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767718.922902:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0e00 x1862022114760064/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767718.922914:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767718.922916:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775767718.922918:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767718.922920:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5096
00000100:00080000:2.0:1775767718.922922:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767718.922924:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767718.922928:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767718.922938:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767718/real 1775767718]  req@ffff9f14852d3480 x1862022114759424/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767743 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767718.922938:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3b80 x1862022114760192/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767718.922959:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767718.922965:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775767718.922967:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767718.922970:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5096
00000100:00080000:2.0:1775767718.922973:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767718.922975:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767718.922978:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775767718.922979:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3480 x1862022114759424/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767743 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767718.922986:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0000 x1862022114760320/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767718.923008:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767718.923015:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767718.923021:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114759424:192.168.202.110@tcp:250:kworker.0
00000100:00100000:0.0:1775767718.923028:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114759552:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767718.923048:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767718.923061:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767718/real 1775767718]  req@ffff9f14852d1c00 x1862022114759552/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767743 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767718.923075:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1c00 x1862022114759552/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767743 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767718.923084:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767718.923087:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767718.923091:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114759552:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767718.923097:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114759680:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767718.923114:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767718.923122:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767718/real 1775767718]  req@ffff9f14852d2300 x1862022114759680/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767743 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767718.923130:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2300 x1862022114759680/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767743 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767718.923140:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767718.923142:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767718.923144:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114759680:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767718.923163:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114759808:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767718.923174:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767718.923180:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767718/real 1775767718]  req@ffff9f14852d3100 x1862022114759808/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767743 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767718.923187:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3100 x1862022114759808/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767743 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767718.923192:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767718.923194:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767718.923196:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114759808:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775767718.923199:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114759936:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767718.923208:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767718.923213:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767718/real 1775767718]  req@ffff9f14852d1880 x1862022114759936/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767743 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767718.923229:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1880 x1862022114759936/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767743 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767718.923237:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767718.923239:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767718.923242:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114759936:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775767718.923246:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114760064:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767718.923257:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767718.923263:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767718/real 1775767718]  req@ffff9f14852d0e00 x1862022114760064/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767743 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767718.923270:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0e00 x1862022114760064/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767743 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767718.923275:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767718.923277:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767718.923278:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114760064:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767718.923282:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114760192:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767718.923291:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767718.923296:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767718/real 1775767718]  req@ffff9f14852d3b80 x1862022114760192/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767743 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767718.923303:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3b80 x1862022114760192/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767743 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767718.923307:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767718.923309:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767718.923311:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114760192:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767718.923314:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114760320:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767718.923322:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767718.923328:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767718/real 1775767718]  req@ffff9f14852d0000 x1862022114760320/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767743 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767718.923335:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0000 x1862022114760320/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767743 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767718.923340:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767718.923342:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767718.923343:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114760320:192.168.202.110@tcp:8:kworker.0
00010000:00010000:0.0:1775767719.178702:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767719.946683:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767720.202712:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767720.970850:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767721.226833:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767721.994733:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767722.250784:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767723.018612:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:0.0:1775767723.274706:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1775767724.042677:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:2.0:1775767724.042677:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:0.0:1775767724.042688:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1775767724.042692:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767724.042701:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5101
00000100:00080000:0.0:1775767724.042711:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767724.042717:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775767724.042754:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3b80 x1862022114760448/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767724.042779:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767724.042815:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775767724.042818:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767724.042824:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5101
00000100:00080000:0.0:1775767724.042829:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00100000:2.0:1775767724.042834:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114760448:192.168.202.110@tcp:250:kworker.0
00000100:00080000:0.0:1775767724.042838:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767724.042853:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767724.042869:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d1f80 x1862022114760576/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767724.042878:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767724.042881:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:0.0:1775767724.042882:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767724.042885:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5101
00000100:00080000:0.0:1775767724.042888:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767724.042891:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767724.042895:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767724.042903:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0380 x1862022114760704/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:2.0:1775767724.042912:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:0.0:1775767724.042916:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767724.042919:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775767724.042920:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767724.042923:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5101
00000100:00080000:0.0:1775767724.042926:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767724.042929:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767724.042932:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775767724.042939:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3480 x1862022114760832/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767724.042945:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767724.042947:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1775767724.042949:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767724.042951:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5101
00000100:00080000:0.0:1775767724.042955:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767724.042957:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767724.042961:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775767724.042962:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767724/real 1775767724]  req@ffff9f14a04d3b80 x1862022114760448/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767749 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767724.042971:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d1180 x1862022114760960/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767724.042983:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767724.042986:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775767724.042987:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767724.042993:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5101
00000100:00100000:2.0:1775767724.042994:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3b80 x1862022114760448/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767749 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767724.042995:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767724.043002:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767724.043007:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00080000:2.0:1775767724.043017:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00100000:0.0:1775767724.043020:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3100 x1862022114761088/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767724.043027:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:0.0:1775767724.043033:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767724.043037:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00100000:2.0:1775767724.043037:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114760448:192.168.202.110@tcp:250:kworker.0
00000100:00080000:0.0:1775767724.043041:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767724.043049:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5101
00000100:00080000:0.0:1775767724.043051:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767724.043053:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767724.043057:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767724.043063:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0e00 x1862022114761216/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767724.043071:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:2.0:1775767724.043071:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114760576:192.168.202.110@tcp:38:kworker.0
00000100:00080000:0.0:1775767724.043076:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775767724.043080:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767724.043084:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5101
00000100:00080000:0.0:1775767724.043087:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767724.043089:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767724.043093:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775767724.043102:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0000 x1862022114761344/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:2.0:1775767724.043111:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767724.043127:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767724/real 1775767724]  req@ffff9f14a04d1f80 x1862022114760576/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767749 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767724.043142:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1f80 x1862022114760576/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767749 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767724.043151:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767724.043155:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767724.043158:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114760576:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767724.043165:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114760704:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775767724.043186:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767724.043196:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767724/real 1775767724]  req@ffff9f14a04d0380 x1862022114760704/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767749 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767724.043216:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0380 x1862022114760704/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767749 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767724.043225:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767724.043228:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767724.043231:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114760704:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767724.043238:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114760832:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775767724.043260:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767724.043271:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767724/real 1775767724]  req@ffff9f14a04d3480 x1862022114760832/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767749 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767724.043283:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3480 x1862022114760832/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767749 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767724.043291:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767724.043295:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767724.043298:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114760832:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775767724.043304:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114760960:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775767724.043324:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767724.043335:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767724/real 1775767724]  req@ffff9f14a04d1180 x1862022114760960/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767749 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767724.043347:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1180 x1862022114760960/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767749 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767724.043362:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767724.043366:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767724.043369:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114760960:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775767724.043376:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114761088:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775767724.043397:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767724.043408:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767724/real 1775767724]  req@ffff9f14a04d3100 x1862022114761088/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767749 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767724.043420:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3100 x1862022114761088/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767749 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767724.043429:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767724.043432:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767724.043436:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114761088:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767724.043458:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114761216:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775767724.043480:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767724.043492:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767724/real 1775767724]  req@ffff9f14a04d0e00 x1862022114761216/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767749 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767724.043504:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0e00 x1862022114761216/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767749 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767724.043512:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767724.043515:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767724.043518:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114761216:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767724.043525:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114761344:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775767724.043544:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767724.043577:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767724/real 1775767724]  req@ffff9f14a04d0000 x1862022114761344/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767749 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767724.043591:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0000 x1862022114761344/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767749 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767724.043599:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767724.043603:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767724.043606:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114761344:192.168.202.110@tcp:8:kworker.0
00010000:00010000:3.0:1775767724.298758:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767725.066683:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767725.322745:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767726.090622:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:0.0:1775767726.346701:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767727.114642:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767727.370705:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767728.138679:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767728.394804:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1775767729.162659:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767729.162670:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00100000:0.0:1775767729.162671:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:2.0:1775767729.162674:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767729.162684:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5106
00000100:00080000:2.0:1775767729.162696:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767729.162702:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775767729.162768:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0380 x1862022114761472/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767729.162796:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767729.162799:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775767729.162801:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767729.162806:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5106
00000100:00080000:2.0:1775767729.162810:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767729.162815:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767729.162824:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767729.162829:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114761472:192.168.202.110@tcp:250:kworker.0
00000100:00100000:2.0:1775767729.162836:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1500 x1862022114761600/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767729.162845:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767729.162849:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775767729.162850:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767729.162854:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5106
00000100:00080000:2.0:1775767729.162857:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767729.162860:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767729.162864:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775767729.162872:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2a00 x1862022114761728/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767729.162880:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767729.162883:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775767729.162895:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767729.162899:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5106
00000100:00080000:2.0:1775767729.162903:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767729.162906:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767729.162911:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775767729.162923:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114761600:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767729.162923:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1180 x1862022114761856/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767729.162933:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767729.162936:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1775767729.162938:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767729.162942:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5106
00000100:00080000:2.0:1775767729.162946:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767729.162949:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775767729.162951:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114761728:192.168.202.110@tcp:38:kworker.0
00000008:00100000:2.0:1775767729.162954:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775767729.162963:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2d80 x1862022114761984/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767729.162973:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1775767729.162976:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114761856:192.168.202.110@tcp:8:kworker.0
00000100:00080000:2.0:1775767729.162977:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775767729.162979:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767729.162982:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5106
00000100:00080000:2.0:1775767729.162986:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767729.162990:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767729.162995:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767729.162998:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114761984:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775767729.163003:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3800 x1862022114762112/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767729.163020:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767729.163024:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775767729.163026:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775767729.163029:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114762112:192.168.202.110@tcp:38:kworker.0
00000100:00080000:2.0:1775767729.163029:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5106
00000100:00080000:2.0:1775767729.163033:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767729.163036:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767729.163041:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775767729.163051:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0a80 x1862022114762240/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767729.163063:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767729.163067:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775767729.163069:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767729.163073:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5106
00000100:00080000:2.0:1775767729.163078:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1775767729.163080:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114762240:192.168.202.110@tcp:38:kworker.0
00000100:00080000:2.0:1775767729.163082:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767729.163087:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775767729.163097:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0700 x1862022114762368/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775767729.163121:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114762368:192.168.202.110@tcp:8:kworker.0
00010000:00010000:3.0:1775767729.418713:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767730.186727:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:0.0:1775767730.442755:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767731.210747:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767731.466841:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767732.234727:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000400:00000100:2.0:1775767732.234887:0:2391:0:(lib-socket.c:454:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.202.110:988
00000400:00000100:2.0:1775767732.234939:0:2391:0:(acceptor.c:111:lnet_connect_console_error()) Connection to 192.168.202.110@tcp at host 192.168.202.110 was unreachable: the network or that node may be down, or Lustre may be misconfigured.
00000800:00000100:2.0:1775767732.234958:0:2391:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:2.0:1775767732.234998:0:2391:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:2.0:1775767732.235012:0:2391:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:2.0:1775767732.235021:0:2391:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:2.0:1775767732.235033:0:2391:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:2.0:1775767732.235049:0:2391:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:2.0:1775767732.235066:0:2391:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000100:00100000:0.0:1775767732.235071:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767729/real 1775767732]  req@ffff9f14852d1180 x1862022114761856/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767754 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000800:00000100:2.0:1775767732.235088:0:2391:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000100:00100000:0.0:1775767732.235130:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1180 x1862022114761856/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767754 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767732.235144:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767732.235150:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767732.235154:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114761856:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775767732.235164:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767729/real 1775767732]  req@ffff9f14852d2d80 x1862022114761984/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767754 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767732.235176:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2d80 x1862022114761984/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767754 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767732.235184:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767732.235187:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767732.235190:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114761984:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775767732.235197:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767729/real 1775767732]  req@ffff9f14852d1500 x1862022114761600/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767754 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767732.235209:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1500 x1862022114761600/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767754 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767732.235217:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767732.235222:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767732.235226:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114761600:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767732.235233:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767729/real 1775767732]  req@ffff9f14852d2a00 x1862022114761728/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767754 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767732.235245:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2a00 x1862022114761728/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767754 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767732.235252:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767732.235257:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767732.235260:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114761728:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767732.235270:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767729/real 1775767732]  req@ffff9f14852d0380 x1862022114761472/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767754 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767732.235283:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0380 x1862022114761472/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767754 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767732.235292:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767732.235296:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767732.235300:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114761472:192.168.202.110@tcp:250:kworker.0
00000100:00100000:0.0:1775767732.235346:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767729/real 1775767732]  req@ffff9f14852d0700 x1862022114762368/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767754 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767732.235359:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0700 x1862022114762368/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767754 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767732.235368:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767732.235372:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767732.235376:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114762368:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775767732.235384:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767729/real 1775767732]  req@ffff9f14852d0a80 x1862022114762240/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767754 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767732.235395:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0a80 x1862022114762240/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767754 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767732.235403:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767732.235406:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767732.235409:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114762240:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767732.235416:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767729/real 1775767732]  req@ffff9f14852d3800 x1862022114762112/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767754 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767732.235427:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3800 x1862022114762112/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767754 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767732.235434:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767732.235438:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767732.235441:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114762112:192.168.202.110@tcp:38:kworker.0
00010000:00010000:3.0:1775767732.490750:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767733.258667:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:0.0:1775767733.514686:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1775767734.282661:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1775767734.282665:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:2.0:1775767734.282674:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1775767734.282678:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767734.282688:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5111
00000100:00080000:2.0:1775767734.282719:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767734.282725:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775767734.282759:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1f80 x1862022114762496/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767734.282784:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767734.282789:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775767734.282790:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767734.282795:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5111
00000100:00080000:2.0:1775767734.282800:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767734.282804:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767734.282816:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775767734.282830:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2680 x1862022114762624/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775767734.282839:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114762496:192.168.202.110@tcp:250:kworker.0
00000100:00080000:2.0:1775767734.282842:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767734.282845:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775767734.282847:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767734.282852:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5111
00000100:00080000:2.0:1775767734.282856:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767734.282859:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767734.282864:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775767734.282875:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3800 x1862022114762752/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767734.282884:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767734.282887:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775767734.282888:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767734.282892:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5111
00000100:00080000:2.0:1775767734.282894:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767734.282897:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000800:00000100:0.0:1775767734.282898:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000008:00100000:2.0:1775767734.282902:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775767734.282911:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0a80 x1862022114762880/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767734.282920:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767734.282923:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1775767734.282925:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767734.282928:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5111
00000100:00080000:2.0:1775767734.282931:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767734.282934:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767734.282939:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775767734.282942:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767734/real 1775767734]  req@ffff9f14852d1f80 x1862022114762496/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767759 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767734.282949:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0700 x1862022114763008/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767734.282958:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1775767734.282959:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1f80 x1862022114762496/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767759 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767734.282961:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775767734.282963:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767734.282967:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5111
00000100:00080000:2.0:1775767734.282970:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767734.282971:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767734.282973:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00080000:0.0:1775767734.282975:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000008:00100000:2.0:1775767734.282978:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767734.282979:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114762496:192.168.202.110@tcp:250:kworker.0
00000100:00100000:2.0:1775767734.282987:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0380 x1862022114763136/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767734.282995:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767734.282998:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775767734.283000:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775767734.283003:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114762624:192.168.202.110@tcp:38:kworker.0
00000100:00080000:2.0:1775767734.283004:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5111
00000100:00080000:2.0:1775767734.283007:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767734.283011:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767734.283016:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775767734.283025:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2a00 x1862022114763264/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1775767734.283026:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:2.0:1775767734.283036:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1775767734.283039:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767734/real 1775767734]  req@ffff9f14852d2680 x1862022114762624/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767759 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767734.283039:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775767734.283041:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767734.283045:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5111
00000100:00080000:2.0:1775767734.283049:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1775767734.283051:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2680 x1862022114762624/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767759 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767734.283053:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767734.283058:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00080000:0.0:1775767734.283059:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767734.283063:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767734.283067:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114762624:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767734.283067:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1500 x1862022114763392/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775767734.283073:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114762752:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767734.283092:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767734.283102:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767734/real 1775767734]  req@ffff9f14852d3800 x1862022114762752/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767759 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767734.283120:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3800 x1862022114762752/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767759 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767734.283128:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767734.283131:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767734.283135:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114762752:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767734.283140:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114762880:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767734.283161:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767734.283170:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767734/real 1775767734]  req@ffff9f14852d0a80 x1862022114762880/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767759 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767734.283181:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0a80 x1862022114762880/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767759 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767734.283189:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767734.283191:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767734.283194:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114762880:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775767734.283199:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114763008:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767734.283217:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767734.283226:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767734/real 1775767734]  req@ffff9f14852d0700 x1862022114763008/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767759 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767734.283238:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0700 x1862022114763008/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767759 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767734.283251:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767734.283255:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767734.283258:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114763008:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775767734.283264:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114763136:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767734.283285:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767734.283296:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767734/real 1775767734]  req@ffff9f14852d0380 x1862022114763136/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767759 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767734.283308:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0380 x1862022114763136/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767759 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767734.283316:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767734.283319:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767734.283323:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114763136:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767734.283344:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114763264:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767734.283368:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767734.283380:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767734/real 1775767734]  req@ffff9f14852d2a00 x1862022114763264/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767759 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767734.283393:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2a00 x1862022114763264/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767759 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767734.283400:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767734.283404:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767734.283408:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114763264:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767734.283414:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114763392:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767734.283436:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767734.283447:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767734/real 1775767734]  req@ffff9f14852d1500 x1862022114763392/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767759 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767734.283460:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1500 x1862022114763392/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767759 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767734.283468:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767734.283471:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767734.283474:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114763392:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775767734.538712:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767735.306731:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767735.562787:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767736.330663:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775767736.586802:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767737.354679:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:0.0:1775767737.610734:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767738.378647:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767738.634792:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1775767739.402703:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767739.402712:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1775767739.402717:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767739.402728:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5116
00000100:00100000:2.0:1775767739.402730:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:1.0:1775767739.402738:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767739.402745:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775767739.402790:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdf80 x1862022114763520/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767739.402821:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767739.402824:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775767739.402825:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767739.402828:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5116
00000100:00080000:1.0:1775767739.402831:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767739.402833:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767739.402848:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767739.402859:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fe300 x1862022114763648/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767739.402867:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767739.402875:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00100000:0.0:1775767739.402876:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114763520:192.168.202.110@tcp:250:kworker.0
00000100:00080000:1.0:1775767739.402877:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767739.402885:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5116
00000100:00080000:1.0:1775767739.402893:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767739.402897:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767739.402900:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767739.402910:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fd880 x1862022114763776/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767739.402918:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767739.402920:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775767739.402921:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767739.402924:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5116
00000100:00080000:1.0:1775767739.402926:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767739.402929:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767739.402933:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775767739.402940:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fea00 x1862022114763904/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767739.402949:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767739.402951:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1775767739.402952:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767739.402954:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5116
00000100:00080000:1.0:1775767739.402967:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767739.402970:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767739.402973:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775767739.402984:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fce00 x1862022114764032/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1775767739.402988:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:1.0:1775767739.403001:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767739.403003:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775767739.403005:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767739.403007:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5116
00000100:00080000:1.0:1775767739.403009:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767739.403013:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767739.403015:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767739.403024:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdc00 x1862022114764160/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767739.403035:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767739.403037:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775767739.403039:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767739.403041:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5116
00000100:00080000:1.0:1775767739.403044:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767739.403049:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767739.403052:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767739.403060:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767739/real 1775767739]  req@ffff9f14ab4fdf80 x1862022114763520/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767764 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767739.403060:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff100 x1862022114764288/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767739.403111:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767739.403115:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775767739.403117:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767739.403122:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5116
00000100:00100000:0.0:1775767739.403128:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdf80 x1862022114763520/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767764 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767739.403130:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767739.403144:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00080000:0.0:1775767739.403153:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000008:00100000:1.0:1775767739.403154:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00080000:0.0:1775767739.403165:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767739.403176:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114763520:192.168.202.110@tcp:250:kworker.0
00000100:00100000:1.0:1775767739.403176:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fc380 x1862022114764416/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775767739.403224:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114763648:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767739.403255:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767739.403271:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767739/real 1775767739]  req@ffff9f14ab4fe300 x1862022114763648/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767764 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767739.403284:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fe300 x1862022114763648/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767764 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767739.403293:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767739.403297:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767739.403301:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114763648:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767739.403308:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114763776:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767739.403330:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767739.403342:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767739/real 1775767739]  req@ffff9f14ab4fd880 x1862022114763776/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767764 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767739.403354:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fd880 x1862022114763776/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767764 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767739.403363:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767739.403367:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767739.403370:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114763776:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767739.403377:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114763904:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767739.403400:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767739.403413:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767739/real 1775767739]  req@ffff9f14ab4fea00 x1862022114763904/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767764 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767739.403425:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fea00 x1862022114763904/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767764 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767739.403435:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767739.403438:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767739.403441:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114763904:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775767739.403448:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114764032:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767739.403468:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767739.403480:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767739/real 1775767739]  req@ffff9f14ab4fce00 x1862022114764032/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767764 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767739.403493:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fce00 x1862022114764032/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767764 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767739.403502:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767739.403505:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767739.403509:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114764032:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775767739.403515:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114764160:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767739.403535:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767739.403547:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767739/real 1775767739]  req@ffff9f14ab4fdc00 x1862022114764160/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767764 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767739.403586:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdc00 x1862022114764160/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767764 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767739.403596:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767739.403600:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767739.403603:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114764160:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767739.403610:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114764288:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767739.403633:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767739.403659:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767739/real 1775767739]  req@ffff9f14ab4ff100 x1862022114764288/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767764 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767739.403672:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff100 x1862022114764288/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767764 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767739.403681:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767739.403684:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767739.403688:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114764288:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767739.403694:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114764416:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767739.403717:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767739.403728:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767739/real 1775767739]  req@ffff9f14ab4fc380 x1862022114764416/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767764 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767739.403741:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fc380 x1862022114764416/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767764 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767739.403755:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767739.403759:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767739.403763:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114764416:192.168.202.110@tcp:8:kworker.0
00010000:00010000:3.0:1775767739.658808:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767740.426656:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:0.0:1775767740.682709:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767741.450681:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767741.706781:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767742.474735:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767742.730761:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767743.498668:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775767743.754715:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767744.522685:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:0.0:1775767744.522735:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767744.522743:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1775767744.522748:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767744.522758:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5122
00000100:00080000:0.0:1775767744.522771:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767744.522778:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775767744.522821:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0a80 x1862022114764544/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767744.522851:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767744.522854:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775767744.522855:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767744.522859:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5122
00000100:00080000:0.0:1775767744.522862:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767744.522865:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767744.522874:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767744.522884:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d2300 x1862022114764672/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775767744.522894:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114764544:192.168.202.110@tcp:250:kworker.0
00000100:00080000:0.0:1775767744.522896:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767744.522907:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:0.0:1775767744.522909:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767744.522913:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5122
00000100:00080000:0.0:1775767744.522916:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767744.522921:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767744.522925:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767744.522934:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d2a00 x1862022114764800/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767744.522941:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767744.522943:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775767744.522944:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767744.522948:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5122
00000100:00080000:0.0:1775767744.522951:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767744.522954:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767744.522958:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775767744.522966:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3800 x1862022114764928/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:2.0:1775767744.522979:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:0.0:1775767744.522981:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767744.522985:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1775767744.522987:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767744.522990:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5122
00000100:00080000:0.0:1775767744.522997:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767744.523000:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767744.523003:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775767744.523012:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d1500 x1862022114765056/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767744.523024:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767744.523029:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00100000:2.0:1775767744.523030:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767744/real 1775767744]  req@ffff9f14a04d0a80 x1862022114764544/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767769 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767744.523031:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767744.523040:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5122
00000100:00080000:0.0:1775767744.523046:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767744.523053:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767744.523064:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767744.523084:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0000 x1862022114765184/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775767744.523091:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0a80 x1862022114764544/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767769 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767744.523104:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767744.523122:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:0.0:1775767744.523128:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767744.523141:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5122
00000100:00080000:0.0:1775767744.523146:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767744.523147:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767744.523152:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00080000:2.0:1775767744.523159:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000008:00100000:0.0:1775767744.523162:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775767744.523168:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114764544:192.168.202.110@tcp:250:kworker.0
00000100:00100000:0.0:1775767744.523177:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0e00 x1862022114765312/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767744.523185:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767744.523187:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775767744.523188:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767744.523193:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5122
00000100:00100000:2.0:1775767744.523200:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114764672:192.168.202.110@tcp:38:kworker.0
00000100:00080000:0.0:1775767744.523201:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767744.523211:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767744.523217:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775767744.523227:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3100 x1862022114765440/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:2.0:1775767744.523238:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767744.523255:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767744/real 1775767744]  req@ffff9f14a04d2300 x1862022114764672/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767769 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767744.523269:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d2300 x1862022114764672/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767769 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767744.523279:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767744.523283:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767744.523287:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114764672:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767744.523294:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114764800:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775767744.523317:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767744.523328:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767744/real 1775767744]  req@ffff9f14a04d2a00 x1862022114764800/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767769 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767744.523337:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d2a00 x1862022114764800/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767769 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767744.523343:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767744.523345:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767744.523347:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114764800:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767744.523352:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114764928:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775767744.523366:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767744.523375:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767744/real 1775767744]  req@ffff9f14a04d3800 x1862022114764928/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767769 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767744.523393:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3800 x1862022114764928/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767769 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767744.523401:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767744.523404:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767744.523419:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114764928:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775767744.523430:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114765056:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775767744.523447:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767744.523455:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767744/real 1775767744]  req@ffff9f14a04d1500 x1862022114765056/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767769 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767744.523463:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1500 x1862022114765056/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767769 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767744.523468:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767744.523471:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767744.523473:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114765056:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775767744.523476:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114765184:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775767744.523487:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767744.523493:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767744/real 1775767744]  req@ffff9f14a04d0000 x1862022114765184/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767769 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767744.523500:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0000 x1862022114765184/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767769 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767744.523509:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767744.523511:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767744.523513:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114765184:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767744.523516:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114765312:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775767744.523526:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767744.523531:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767744/real 1775767744]  req@ffff9f14a04d0e00 x1862022114765312/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767769 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767744.523538:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0e00 x1862022114765312/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767769 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767744.523542:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767744.523544:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767744.523546:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114765312:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767744.523584:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114765440:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775767744.523612:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767744.523620:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767744/real 1775767744]  req@ffff9f14a04d3100 x1862022114765440/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767769 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767744.523628:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3100 x1862022114765440/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767769 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767744.523633:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767744.523636:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767744.523638:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114765440:192.168.202.110@tcp:8:kworker.0
00010000:00010000:0.0:1775767744.778722:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767745.546675:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767745.802784:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767746.570694:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767746.826790:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767747.594670:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:0.0:1775767747.850736:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767748.618715:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767748.874804:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767749.642636:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:0.0:1775767749.642677:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767749.642687:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1775767749.642691:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767749.642701:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5127
00000100:00080000:0.0:1775767749.642714:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767749.642721:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775767749.642763:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d1180 x1862022114765568/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767749.642790:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767749.642794:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775767749.642795:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767749.642799:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5127
00000100:00080000:0.0:1775767749.642802:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767749.642806:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767749.642815:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767749.642824:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3480 x1862022114765696/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775767749.642831:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114765568:192.168.202.110@tcp:250:kworker.0
00000100:00080000:0.0:1775767749.642856:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767749.642858:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:0.0:1775767749.642859:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767749.642864:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5127
00000100:00080000:0.0:1775767749.642866:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767749.642868:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767749.642872:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767749.642881:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0380 x1862022114765824/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767749.642887:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767749.642890:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775767749.642892:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767749.642895:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5127
00000100:00080000:0.0:1775767749.642898:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767749.642900:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767749.642905:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775767749.642912:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d1f80 x1862022114765952/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767749.642917:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767749.642919:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1775767749.642921:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000800:00000100:2.0:1775767749.642929:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:0.0:1775767749.642931:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5127
00000100:00080000:0.0:1775767749.642937:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767749.642940:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767749.642945:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775767749.642958:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3b80 x1862022114766080/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767749.642966:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767749.642969:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775767749.642971:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767749.642978:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5127
00000100:00080000:0.0:1775767749.642983:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:2.0:1775767749.642987:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767749/real 1775767749]  req@ffff9f14a04d1180 x1862022114765568/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767774 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767749.642988:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767749.642999:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775767749.643021:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1180 x1862022114765568/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767774 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767749.643031:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3100 x1862022114766208/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767749.643035:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767749.643051:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:0.0:1775767749.643062:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:2.0:1775767749.643064:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114765568:192.168.202.110@tcp:250:kworker.0
00000100:00080000:0.0:1775767749.643072:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:0.0:1775767749.643079:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767749.643086:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5127
00000100:00080000:0.0:1775767749.643091:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767749.643097:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767749.643103:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775767749.643109:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114765696:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767749.643121:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0e00 x1862022114766336/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767749.643135:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000800:00000100:2.0:1775767749.643144:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:0.0:1775767749.643148:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775767749.643150:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767749.643156:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5127
00000100:00080000:0.0:1775767749.643162:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767749.643166:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767749.643171:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775767749.643172:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767749/real 1775767749]  req@ffff9f14a04d3480 x1862022114765696/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767774 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767749.643184:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0000 x1862022114766464/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775767749.643217:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3480 x1862022114765696/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767774 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767749.643233:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767749.643238:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767749.643241:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114765696:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767749.643249:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114765824:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775767749.643274:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767749.643284:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767749/real 1775767749]  req@ffff9f14a04d0380 x1862022114765824/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767774 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767749.643292:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0380 x1862022114765824/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767774 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767749.643298:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767749.643300:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767749.643302:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114765824:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767749.643307:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114765952:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775767749.643319:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767749.643324:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767749/real 1775767749]  req@ffff9f14a04d1f80 x1862022114765952/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767774 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767749.643332:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1f80 x1862022114765952/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767774 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767749.643337:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767749.643339:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767749.643340:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114765952:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775767749.643344:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114766080:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775767749.643354:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767749.643359:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767749/real 1775767749]  req@ffff9f14a04d3b80 x1862022114766080/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767774 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767749.643367:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3b80 x1862022114766080/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767774 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767749.643371:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767749.643373:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767749.643375:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114766080:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775767749.643379:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114766208:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775767749.643390:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767749.643395:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767749/real 1775767749]  req@ffff9f14a04d3100 x1862022114766208/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767774 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767749.643402:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3100 x1862022114766208/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767774 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767749.643407:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767749.643409:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767749.643411:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114766208:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767749.643426:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114766336:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775767749.643436:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767749.643441:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767749/real 1775767749]  req@ffff9f14a04d0e00 x1862022114766336/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767774 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767749.643453:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0e00 x1862022114766336/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767774 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767749.643458:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767749.643460:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767749.643461:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114766336:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767749.643465:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114766464:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775767749.643474:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767749.643480:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767749/real 1775767749]  req@ffff9f14a04d0000 x1862022114766464/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767774 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767749.643487:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0000 x1862022114766464/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767774 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767749.643492:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767749.643493:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767749.643495:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114766464:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775767749.898744:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767750.666660:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767750.922714:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767751.690652:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:0.0:1775767751.946656:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767752.714655:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767752.970738:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767753.738663:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767753.994694:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767754.762656:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:0.0:1775767754.762681:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767754.762686:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1775767754.762690:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767754.762699:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5132
00000100:00080000:0.0:1775767754.762710:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767754.762715:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775767754.762757:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d1500 x1862022114766592/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767754.762777:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767754.762780:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775767754.762781:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767754.762784:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5132
00000100:00080000:0.0:1775767754.762786:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767754.762790:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767754.762799:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775767754.762809:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114766592:192.168.202.110@tcp:250:kworker.0
00000100:00100000:0.0:1775767754.762810:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3800 x1862022114766720/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767754.762827:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767754.762830:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:0.0:1775767754.762831:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767754.762834:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5132
00000100:00080000:0.0:1775767754.762837:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767754.762839:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767754.762843:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767754.762850:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d2a00 x1862022114766848/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767754.762857:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767754.762860:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775767754.762861:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767754.762863:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5132
00000100:00080000:0.0:1775767754.762866:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000800:00000100:2.0:1775767754.762867:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:0.0:1775767754.762871:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767754.762875:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775767754.762883:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d2300 x1862022114766976/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767754.762889:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767754.762892:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1775767754.762894:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767754.762897:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5132
00000100:00080000:0.0:1775767754.762900:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00100000:2.0:1775767754.762904:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767754/real 1775767754]  req@ffff9f14a04d1500 x1862022114766592/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767779 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767754.762907:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767754.762919:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775767754.762924:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1500 x1862022114766592/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767779 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767754.762929:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0a80 x1862022114767104/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767754.762935:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767754.762939:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767754.762942:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:0.0:1775767754.762943:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775767754.762946:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:2.0:1775767754.762947:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114766592:192.168.202.110@tcp:250:kworker.0
00000100:00080000:0.0:1775767754.762951:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5132
00000100:00080000:0.0:1775767754.762955:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767754.762957:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767754.762961:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767754.762968:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d1500 x1862022114767232/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775767754.762972:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114766720:192.168.202.110@tcp:38:kworker.0
00000100:00080000:0.0:1775767754.762985:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767754.762988:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:0.0:1775767754.762990:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767754.762994:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5132
00000800:00000100:2.0:1775767754.762998:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:0.0:1775767754.763000:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767754.763004:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767754.763009:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775767754.763011:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767754/real 1775767754]  req@ffff9f14a04d3800 x1862022114766720/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767779 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767754.763019:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0000 x1862022114767360/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775767754.763023:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3800 x1862022114766720/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767779 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767754.763041:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767754.763042:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767754.763043:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775767754.763048:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767754.763050:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00080000:0.0:1775767754.763055:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5132
00000100:00100000:2.0:1775767754.763055:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114766720:192.168.202.110@tcp:38:kworker.0
00000100:00080000:0.0:1775767754.763063:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:2.0:1775767754.763065:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114766848:192.168.202.110@tcp:38:kworker.0
00000100:00080000:0.0:1775767754.763072:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767754.763078:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:2.0:1775767754.763083:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767754.763095:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767754/real 1775767754]  req@ffff9f14a04d2a00 x1862022114766848/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767779 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767754.763096:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0e00 x1862022114767488/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775767754.763119:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d2a00 x1862022114766848/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767779 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767754.763130:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767754.763132:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767754.763135:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114766848:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767754.763139:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114766976:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775767754.763151:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767754.763157:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767754/real 1775767754]  req@ffff9f14a04d2300 x1862022114766976/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767779 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767754.763164:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d2300 x1862022114766976/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767779 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767754.763177:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767754.763179:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767754.763181:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114766976:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775767754.763185:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114767104:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775767754.763196:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767754.763202:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767754/real 1775767754]  req@ffff9f14a04d0a80 x1862022114767104/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767779 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767754.763209:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0a80 x1862022114767104/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767779 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767754.763213:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767754.763215:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767754.763218:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114767104:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775767754.763230:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114767232:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775767754.763240:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767754.763246:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767754/real 1775767754]  req@ffff9f14a04d1500 x1862022114767232/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767779 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767754.763253:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1500 x1862022114767232/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767779 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767754.763258:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767754.763260:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767754.763261:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114767232:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767754.763265:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114767360:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775767754.763274:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767754.763285:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767754/real 1775767754]  req@ffff9f14a04d0000 x1862022114767360/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767779 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767754.763293:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0000 x1862022114767360/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767779 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767754.763298:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767754.763300:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767754.763302:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114767360:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767754.763306:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114767488:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775767754.763316:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767754.763322:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767754/real 1775767754]  req@ffff9f14a04d0e00 x1862022114767488/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767779 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767754.763329:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0e00 x1862022114767488/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767779 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767754.763334:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767754.763335:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767754.763337:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114767488:192.168.202.110@tcp:8:kworker.0
00010000:00010000:0.0:1775767755.018684:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767755.786672:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767756.042736:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767756.810655:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775767757.066765:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767757.834637:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767758.090730:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767758.858688:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:0.0:1775767759.114690:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1775767759.882657:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767759.882667:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00100000:2.0:1775767759.882668:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:0.0:1775767759.882672:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767759.882682:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5137
00000100:00080000:0.0:1775767759.882693:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767759.882700:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775767759.882736:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3100 x1862022114767616/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767759.882758:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767759.882761:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775767759.882763:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767759.882766:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5137
00000100:00080000:0.0:1775767759.882769:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767759.882773:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767759.882781:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775767759.882785:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114767616:192.168.202.110@tcp:250:kworker.0
00000100:00100000:0.0:1775767759.882793:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3b80 x1862022114767744/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767759.882801:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767759.882803:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:0.0:1775767759.882805:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767759.882808:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5137
00000100:00080000:0.0:1775767759.882810:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767759.882813:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767759.882815:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767759.882822:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d1f80 x1862022114767872/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767759.882828:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767759.882830:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775767759.882831:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767759.882834:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5137
00000100:00080000:0.0:1775767759.882837:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000800:00000100:2.0:1775767759.882839:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:0.0:1775767759.882842:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767759.882846:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775767759.882854:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0380 x1862022114768000/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767759.882860:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767759.882862:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1775767759.882863:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767759.882866:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5137
00000100:00080000:0.0:1775767759.882868:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767759.882870:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767759.882874:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775767759.882878:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767759/real 1775767759]  req@ffff9f14a04d3100 x1862022114767616/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767784 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767759.882882:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3480 x1862022114768128/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767759.882888:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767759.882890:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775767759.882891:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767759.882898:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5137
00000100:00080000:0.0:1775767759.882904:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767759.882907:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767759.882911:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775767759.882914:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3100 x1862022114767616/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767784 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767759.882920:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d1180 x1862022114768256/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767759.882929:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767759.882937:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:0.0:1775767759.882942:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:2.0:1775767759.882944:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114767616:192.168.202.110@tcp:250:kworker.0
00000100:00080000:0.0:1775767759.882945:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:0.0:1775767759.882948:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767759.882953:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5137
00000100:00080000:0.0:1775767759.882955:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767759.882958:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767759.882961:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775767759.882970:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114767744:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767759.882971:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3100 x1862022114768384/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767759.882984:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767759.882986:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775767759.882987:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767759.882990:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5137
00000100:00080000:0.0:1775767759.882993:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767759.882998:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000800:00000100:2.0:1775767759.882998:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000008:00100000:0.0:1775767759.883005:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775767759.883016:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0e00 x1862022114768512/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775767759.883016:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767759/real 1775767759]  req@ffff9f14a04d3b80 x1862022114767744/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767784 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767759.883068:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3b80 x1862022114767744/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767784 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767759.883078:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767759.883082:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767759.883086:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114767744:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767759.883093:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114767872:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775767759.883117:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767759.883129:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767759/real 1775767759]  req@ffff9f14a04d1f80 x1862022114767872/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767784 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767759.883142:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1f80 x1862022114767872/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767784 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767759.883151:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767759.883154:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767759.883158:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114767872:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767759.883165:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114768000:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775767759.883187:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767759.883199:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767759/real 1775767759]  req@ffff9f14a04d0380 x1862022114768000/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767784 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767759.883211:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0380 x1862022114768000/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767784 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767759.883226:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767759.883230:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767759.883233:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114768000:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775767759.883240:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114768128:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775767759.883260:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767759.883271:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767759/real 1775767759]  req@ffff9f14a04d3480 x1862022114768128/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767784 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767759.883282:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3480 x1862022114768128/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767784 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767759.883290:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767759.883293:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767759.883296:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114768128:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775767759.883302:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114768256:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775767759.883320:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767759.883331:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767759/real 1775767759]  req@ffff9f14a04d1180 x1862022114768256/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767784 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767759.883344:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1180 x1862022114768256/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767784 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767759.883353:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767759.883356:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767759.883360:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114768256:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767759.883382:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114768384:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775767759.883413:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767759.883425:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767759/real 1775767759]  req@ffff9f14a04d3100 x1862022114768384/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767784 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767759.883438:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3100 x1862022114768384/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767784 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767759.883461:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767759.883465:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767759.883468:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114768384:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767759.883475:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114768512:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775767759.883497:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767759.883509:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767759/real 1775767759]  req@ffff9f14a04d0e00 x1862022114768512/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767784 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767759.883522:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0e00 x1862022114768512/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767784 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767759.883530:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767759.883534:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767759.883537:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114768512:192.168.202.110@tcp:8:kworker.0
00010000:00010000:3.0:1775767760.138740:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767760.906637:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767761.162711:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767761.930617:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:0.0:1775767762.186661:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767762.954677:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767763.210702:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767763.978671:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775767764.234689:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767765.002636:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775767765.002676:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767765.002682:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775767765.002686:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767765.002694:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5142
00000100:00080000:3.0:1775767765.002703:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767765.002709:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775767765.002743:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747b80 x1862022114768640/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767765.002766:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767765.002769:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775767765.002771:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767765.002773:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5142
00000100:00080000:3.0:1775767765.002776:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767765.002778:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767765.002786:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767765.002796:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746300 x1862022114768768/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767765.002804:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:2.0:1775767765.002806:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114768640:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775767765.002807:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775767765.002810:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767765.002818:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5142
00000100:00080000:3.0:1775767765.002820:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767765.002822:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767765.002825:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767765.002835:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745880 x1862022114768896/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767765.002841:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767765.002843:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775767765.002844:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767765.002847:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5142
00000100:00080000:3.0:1775767765.002856:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767765.002859:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767765.002862:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:2.0:1775767765.002870:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767765.002871:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747480 x1862022114769024/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767765.002879:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767765.002882:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775767765.002883:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767765.002887:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5142
00000100:00080000:3.0:1775767765.002890:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767765.002892:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767765.002894:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775767765.002901:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744a80 x1862022114769152/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767765.002907:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:2.0:1775767765.002909:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767765/real 1775767765]  req@ffff9f1485747b80 x1862022114768640/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767790 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767765.002912:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775767765.002915:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767765.002924:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5142
00000100:00080000:3.0:1775767765.002927:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767765.002930:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767765.002934:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775767765.002936:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747b80 x1862022114768640/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767790 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767765.002944:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746a00 x1862022114769280/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767765.002947:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767765.002956:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:3.0:1775767765.002956:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767765.002961:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00100000:2.0:1775767765.002962:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114768640:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775767765.002963:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767765.002972:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5142
00000100:00080000:3.0:1775767765.002974:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767765.002977:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767765.002979:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767765.002985:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744380 x1862022114769408/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767765.002992:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:2.0:1775767765.002995:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114768768:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775767765.002995:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775767765.002999:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767765.003006:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5142
00000100:00080000:3.0:1775767765.003008:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767765.003010:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767765.003013:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775767765.003022:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744e00 x1862022114769536/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:2.0:1775767765.003034:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767765.003052:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767765/real 1775767765]  req@ffff9f1485746300 x1862022114768768/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767790 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767765.003066:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746300 x1862022114768768/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767790 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767765.003074:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767765.003078:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767765.003081:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114768768:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767765.003086:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114768896:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775767765.003102:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767765.003109:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767765/real 1775767765]  req@ffff9f1485745880 x1862022114768896/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767790 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767765.003118:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745880 x1862022114768896/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767790 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767765.003123:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767765.003125:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767765.003127:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114768896:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767765.003131:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114769024:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775767765.003141:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767765.003146:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767765/real 1775767765]  req@ffff9f1485747480 x1862022114769024/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767790 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767765.003154:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747480 x1862022114769024/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767790 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767765.003159:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767765.003160:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767765.003162:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114769024:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775767765.003165:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114769152:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775767765.003174:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767765.003179:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767765/real 1775767765]  req@ffff9f1485744a80 x1862022114769152/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767790 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767765.003185:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744a80 x1862022114769152/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767790 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767765.003195:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767765.003197:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767765.003199:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114769152:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775767765.003202:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114769280:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775767765.003211:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767765.003216:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767765/real 1775767765]  req@ffff9f1485746a00 x1862022114769280/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767790 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767765.003223:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746a00 x1862022114769280/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767790 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767765.003228:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767765.003230:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767765.003231:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114769280:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767765.003243:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114769408:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775767765.003253:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767765.003259:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767765/real 1775767765]  req@ffff9f1485744380 x1862022114769408/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767790 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767765.003266:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744380 x1862022114769408/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767790 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767765.003270:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767765.003272:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767765.003274:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114769408:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767765.003278:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114769536:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775767765.003287:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767765.003292:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767765/real 1775767765]  req@ffff9f1485744e00 x1862022114769536/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767790 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767765.003298:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744e00 x1862022114769536/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767790 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767765.003303:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767765.003305:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767765.003307:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114769536:192.168.202.110@tcp:8:kworker.0
00010000:00010000:3.0:1775767765.258711:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767766.026692:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:0.0:1775767766.282665:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767767.050633:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767767.306725:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767768.074653:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767768.330758:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767769.098641:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:0.0:1775767769.354671:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767770.122678:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775767770.122683:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767770.122696:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775767770.122702:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767770.122715:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5147
00000100:00080000:3.0:1775767770.122728:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767770.122736:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775767770.122779:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745500 x1862022114769664/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767770.122807:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767770.122811:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775767770.122813:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767770.122818:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5147
00000100:00080000:3.0:1775767770.122822:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767770.122827:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767770.122838:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775767770.122850:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114769664:192.168.202.110@tcp:250:kworker.0
00000100:00100000:3.0:1775767770.122855:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746680 x1862022114769792/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767770.122869:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767770.122874:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775767770.122876:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767770.122881:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5147
00000100:00080000:3.0:1775767770.122896:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767770.122901:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767770.122907:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000800:00000100:2.0:1775767770.122910:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767770.122926:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745c00 x1862022114769920/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767770.122938:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767770.122941:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775767770.122943:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:2.0:1775767770.122946:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767770/real 1775767770]  req@ffff9f1485745500 x1862022114769664/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767795 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767770.122951:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5147
00000100:00080000:3.0:1775767770.122960:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767770.122966:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775767770.122968:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745500 x1862022114769664/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767795 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:3.0:1775767770.122976:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00080000:2.0:1775767770.122980:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767770.122984:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767770.122987:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114769664:192.168.202.110@tcp:250:kworker.0
00000100:00100000:3.0:1775767770.122993:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745180 x1862022114770048/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767770.123005:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:2.0:1775767770.123009:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114769792:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775767770.123010:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775767770.123015:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767770.123021:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5147
00000100:00080000:3.0:1775767770.123025:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767770.123030:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767770.123036:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:2.0:1775767770.123043:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767770.123053:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747100 x1862022114770176/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775767770.123057:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767770/real 1775767770]  req@ffff9f1485746680 x1862022114769792/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767795 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767770.123086:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767770.123089:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00100000:2.0:1775767770.123092:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746680 x1862022114769792/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767795 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767770.123093:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767770.123100:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5147
00000100:00080000:2.0:1775767770.123108:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767770.123109:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767770.123114:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00080000:3.0:1775767770.123118:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775767770.123119:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114769792:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767770.123129:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114769920:192.168.202.110@tcp:38:kworker.0
00000008:00100000:3.0:1775767770.123129:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767770.123149:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744000 x1862022114770304/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:2.0:1775767770.123150:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767770.123159:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767770/real 1775767770]  req@ffff9f1485745c00 x1862022114769920/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767795 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767770.123166:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767770.123171:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00100000:2.0:1775767770.123172:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745c00 x1862022114769920/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767795 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767770.123176:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767770.123186:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767770.123188:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5147
00000100:00080000:2.0:1775767770.123192:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767770.123195:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114769920:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775767770.123196:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:2.0:1775767770.123201:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114770048:192.168.202.110@tcp:8:kworker.0
00000100:00080000:3.0:1775767770.123205:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767770.123213:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000800:00000100:2.0:1775767770.123218:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767770.123225:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767770/real 1775767770]  req@ffff9f1485745180 x1862022114770048/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767795 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767770.123231:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747800 x1862022114770432/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775767770.123239:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745180 x1862022114770048/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767795 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767770.123250:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767770.123252:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767770.123258:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00080000:3.0:1775767770.123259:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00100000:2.0:1775767770.123263:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114770048:192.168.202.110@tcp:8:kworker.0
00000100:00080000:3.0:1775767770.123264:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767770.123272:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5147
00000100:00100000:2.0:1775767770.123277:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114770176:192.168.202.110@tcp:8:kworker.0
00000100:00080000:3.0:1775767770.123278:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767770.123285:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767770.123291:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:2.0:1775767770.123294:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767770.123303:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767770/real 1775767770]  req@ffff9f1485747100 x1862022114770176/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767795 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767770.123311:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745f80 x1862022114770560/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775767770.123313:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747100 x1862022114770176/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767795 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767770.123342:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767770.123360:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767770.123365:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114770176:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775767770.123373:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114770304:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775767770.123399:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767770.123413:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767770/real 1775767770]  req@ffff9f1485744000 x1862022114770304/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767795 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767770.123423:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744000 x1862022114770304/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767795 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767770.123430:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767770.123434:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767770.123438:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114770304:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767770.123444:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114770432:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775767770.123464:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767770.123470:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767770/real 1775767770]  req@ffff9f1485747800 x1862022114770432/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767795 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767770.123478:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747800 x1862022114770432/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767795 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767770.123484:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767770.123488:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767770.123490:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114770432:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767770.123501:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114770560:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775767770.123522:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767770.123535:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767770/real 1775767770]  req@ffff9f1485745f80 x1862022114770560/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767795 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767770.123549:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745f80 x1862022114770560/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767795 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767770.123593:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767770.123597:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767770.123600:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114770560:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775767770.378750:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767771.146656:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775767771.402680:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767772.170700:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767772.426789:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767773.194704:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:0.0:1775767773.450651:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767774.218667:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767774.474695:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1775767775.242640:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1775767775.242641:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775767775.242648:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775767775.242651:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767775.242662:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5152
00000100:00080000:3.0:1775767775.242672:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767775.242679:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775767775.242725:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744700 x1862022114770688/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767775.242746:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767775.242749:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775767775.242750:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767775.242753:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5152
00000100:00080000:3.0:1775767775.242756:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767775.242759:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767775.242768:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775767775.242780:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114770688:192.168.202.110@tcp:250:kworker.0
00000100:00100000:3.0:1775767775.242781:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745f80 x1862022114770816/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767775.242800:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767775.242803:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775767775.242804:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767775.242807:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5152
00000100:00080000:3.0:1775767775.242809:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767775.242812:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767775.242815:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767775.242823:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747800 x1862022114770944/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767775.242829:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767775.242832:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775767775.242833:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767775.242835:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5152
00000100:00080000:3.0:1775767775.242838:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767775.242840:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767775.242843:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:2.0:1775767775.242845:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767775.242853:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744000 x1862022114771072/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767775.242860:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767775.242863:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775767775.242864:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767775.242866:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5152
00000100:00080000:3.0:1775767775.242869:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767775.242872:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767775.242876:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775767775.242883:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767775/real 1775767775]  req@ffff9f1485744700 x1862022114770688/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767800 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767775.242885:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747100 x1862022114771200/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767775.242910:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767775.242913:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775767775.242915:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:2.0:1775767775.242917:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744700 x1862022114770688/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767800 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767775.242921:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5152
00000100:00080000:3.0:1775767775.242926:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767775.242933:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00080000:2.0:1775767775.242934:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000008:00100000:3.0:1775767775.242936:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00080000:2.0:1775767775.242944:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767775.242946:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745180 x1862022114771328/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775767775.242954:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114770688:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775767775.242955:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767775.242962:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775767775.242965:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767775.242968:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5152
00000100:00080000:3.0:1775767775.242970:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767775.242973:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767775.242975:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767775.242981:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745c00 x1862022114771456/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775767775.242988:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114770816:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775767775.243001:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767775.243003:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775767775.243004:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767775.243008:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5152
00000100:00080000:3.0:1775767775.243010:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767775.243013:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767775.243016:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:2.0:1775767775.243024:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767775.243026:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746680 x1862022114771584/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775767775.243039:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767775/real 1775767775]  req@ffff9f1485745f80 x1862022114770816/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767800 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767775.243048:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745f80 x1862022114770816/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767800 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767775.243054:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767775.243056:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767775.243058:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114770816:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767775.243062:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114770944:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775767775.243074:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767775.243079:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767775/real 1775767775]  req@ffff9f1485747800 x1862022114770944/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767800 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767775.243091:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747800 x1862022114770944/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767800 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767775.243096:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767775.243097:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767775.243099:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114770944:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767775.243103:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114771072:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775767775.243113:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767775.243118:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767775/real 1775767775]  req@ffff9f1485744000 x1862022114771072/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767800 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767775.243125:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744000 x1862022114771072/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767800 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767775.243130:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767775.243132:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767775.243134:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114771072:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775767775.243137:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114771200:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775767775.243146:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767775.243151:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767775/real 1775767775]  req@ffff9f1485747100 x1862022114771200/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767800 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767775.243158:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747100 x1862022114771200/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767800 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767775.243167:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767775.243170:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767775.243172:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114771200:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775767775.243175:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114771328:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775767775.243184:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767775.243189:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767775/real 1775767775]  req@ffff9f1485745180 x1862022114771328/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767800 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767775.243195:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745180 x1862022114771328/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767800 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767775.243201:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767775.243202:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767775.243204:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114771328:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767775.243217:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114771456:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775767775.243226:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767775.243244:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767775/real 1775767775]  req@ffff9f1485745c00 x1862022114771456/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767800 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767775.243251:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745c00 x1862022114771456/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767800 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767775.243256:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767775.243258:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767775.243260:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114771456:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767775.243264:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114771584:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775767775.243275:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767775.243280:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767775/real 1775767775]  req@ffff9f1485746680 x1862022114771584/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767800 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767775.243287:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746680 x1862022114771584/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767800 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767775.243292:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767775.243318:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767775.243321:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114771584:192.168.202.110@tcp:8:kworker.0
00010000:00010000:3.0:1775767775.498754:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767776.266675:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:0.0:1775767776.522710:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767777.290631:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767777.546704:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767778.314678:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775767778.570712:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767779.338661:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767779.594711:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767780.362649:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:0.0:1775767780.362682:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767780.362689:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1775767780.362701:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767780.362711:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5157
00000100:00080000:0.0:1775767780.362723:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767780.362730:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775767780.362771:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0000 x1862022114771712/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767780.362798:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767780.362801:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775767780.362803:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767780.362807:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5157
00000100:00080000:0.0:1775767780.362811:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767780.362817:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775767780.362825:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114771712:192.168.202.110@tcp:250:kworker.0
00000008:00100000:0.0:1775767780.362827:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767780.362844:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d1500 x1862022114771840/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767780.362854:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767780.362858:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:0.0:1775767780.362860:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767780.362863:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5157
00000100:00080000:0.0:1775767780.362868:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767780.362872:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767780.362877:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000800:00000100:2.0:1775767780.362882:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767780.362890:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0a80 x1862022114771968/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767780.362901:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767780.362904:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775767780.362906:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767780.362911:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5157
00000100:00080000:0.0:1775767780.362915:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00100000:2.0:1775767780.362916:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767780/real 1775767780]  req@ffff9f14a04d0000 x1862022114771712/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767805 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767780.362921:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767780.362926:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775767780.362936:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d2300 x1862022114772096/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775767780.362943:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0000 x1862022114771712/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767805 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767780.362944:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767780.362951:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1775767780.362955:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767780.362963:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5157
00000100:00080000:2.0:1775767780.362963:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767780.362965:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767780.362970:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00080000:2.0:1775767780.362970:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000008:00100000:0.0:1775767780.362975:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775767780.362976:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114771712:192.168.202.110@tcp:250:kworker.0
00000100:00100000:0.0:1775767780.362985:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d2a00 x1862022114772224/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767780.362992:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767780.362994:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775767780.362996:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767780.362999:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5157
00000100:00080000:0.0:1775767780.363002:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:2.0:1775767780.363002:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114771840:192.168.202.110@tcp:38:kworker.0
00000100:00080000:0.0:1775767780.363009:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767780.363013:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767780.363022:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3800 x1862022114772352/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:2.0:1775767780.363029:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:0.0:1775767780.363038:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767780.363041:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:0.0:1775767780.363044:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:2.0:1775767780.363045:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767780/real 1775767780]  req@ffff9f14a04d1500 x1862022114771840/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767805 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767780.363052:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5157
00000100:00080000:0.0:1775767780.363058:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767780.363063:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775767780.363067:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1500 x1862022114771840/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767805 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:0.0:1775767780.363068:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00080000:2.0:1775767780.363078:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00100000:0.0:1775767780.363081:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0000 x1862022114772480/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767780.363084:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767780.363091:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114771840:192.168.202.110@tcp:38:kworker.0
00000100:00080000:0.0:1775767780.363100:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:2.0:1775767780.363102:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114771968:192.168.202.110@tcp:38:kworker.0
00000100:00080000:0.0:1775767780.363105:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775767780.363108:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767780.363112:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5157
00000100:00080000:0.0:1775767780.363114:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767780.363117:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767780.363120:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:2.0:1775767780.363128:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767780.363130:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0e00 x1862022114772608/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775767780.363143:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767780/real 1775767780]  req@ffff9f14a04d0a80 x1862022114771968/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767805 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767780.363158:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0a80 x1862022114771968/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767805 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767780.363167:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767780.363171:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767780.363174:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114771968:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767780.363181:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114772096:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775767780.363198:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767780.363206:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767780/real 1775767780]  req@ffff9f14a04d2300 x1862022114772096/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767805 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767780.363214:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d2300 x1862022114772096/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767805 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767780.363219:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767780.363221:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767780.363223:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114772096:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775767780.363227:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114772224:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775767780.363236:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767780.363241:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767780/real 1775767780]  req@ffff9f14a04d2a00 x1862022114772224/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767805 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767780.363273:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d2a00 x1862022114772224/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767805 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767780.363278:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767780.363280:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767780.363282:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114772224:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775767780.363296:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114772352:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775767780.363312:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767780.363322:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767780/real 1775767780]  req@ffff9f14a04d3800 x1862022114772352/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767805 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767780.363333:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3800 x1862022114772352/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767805 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767780.363357:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767780.363364:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767780.363366:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114772352:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767780.363371:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114772480:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775767780.363385:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767780.363392:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767780/real 1775767780]  req@ffff9f14a04d0000 x1862022114772480/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767805 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767780.363400:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0000 x1862022114772480/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767805 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767780.363405:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767780.363407:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767780.363409:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114772480:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767780.363412:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114772608:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775767780.363421:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767780.363426:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767780/real 1775767780]  req@ffff9f14a04d0e00 x1862022114772608/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767805 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767780.363433:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0e00 x1862022114772608/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767805 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767780.363438:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767780.363440:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767780.363442:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114772608:192.168.202.110@tcp:8:kworker.0
00010000:00010000:0.0:1775767780.618762:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767781.386610:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767781.642704:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767782.410664:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767782.666730:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767783.434696:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:0.0:1775767783.690662:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767784.458625:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767784.714724:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767785.482682:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775767785.482698:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767785.482706:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775767785.482711:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767785.482722:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5162
00000100:00080000:3.0:1775767785.482732:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767785.482738:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775767785.482773:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745500 x1862022114772736/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767785.482797:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767785.482800:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775767785.482802:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767785.482805:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5162
00000100:00080000:3.0:1775767785.482807:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767785.482810:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767785.482819:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767785.482826:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744e00 x1862022114772864/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767785.482833:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767785.482835:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775767785.482836:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767785.482839:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5162
00000100:00080000:3.0:1775767785.482840:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767785.482842:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767785.482844:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767785.482850:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744380 x1862022114772992/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767785.482856:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767785.482858:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775767785.482859:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767785.482862:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5162
00000100:00080000:3.0:1775767785.482864:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767785.482866:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767785.482869:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775767785.482877:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746a00 x1862022114773120/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767785.482883:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:2.0:1775767785.482884:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114772736:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775767785.482885:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775767785.482889:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767785.482895:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5162
00000100:00080000:3.0:1775767785.482897:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767785.482899:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767785.482901:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775767785.482909:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744a80 x1862022114773248/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767785.482915:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767785.482917:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775767785.482918:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767785.482921:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5162
00000100:00080000:3.0:1775767785.482923:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767785.482927:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767785.482930:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767785.482936:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747480 x1862022114773376/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767785.482942:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767785.482943:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775767785.482945:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767785.482947:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5162
00000800:00000100:2.0:1775767785.482952:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775767785.482953:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767785.482958:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767785.482962:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767785.482971:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745880 x1862022114773504/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767785.482977:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767785.482979:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775767785.482980:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767785.482983:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5162
00000100:00080000:3.0:1775767785.482986:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767785.482989:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767785.482992:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775767785.482993:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767785/real 1775767785]  req@ffff9f1485745500 x1862022114772736/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767810 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767785.483002:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746300 x1862022114773632/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775767785.483013:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745500 x1862022114772736/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767810 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767785.483021:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767785.483025:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767785.483029:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114772736:192.168.202.110@tcp:250:kworker.0
00000100:00100000:2.0:1775767785.483034:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114772864:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775767785.483049:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767785.483056:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767785/real 1775767785]  req@ffff9f1485744e00 x1862022114772864/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767810 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767785.483064:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744e00 x1862022114772864/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767810 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767785.483069:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767785.483071:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767785.483074:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114772864:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767785.483079:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114772992:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775767785.483088:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767785.483094:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767785/real 1775767785]  req@ffff9f1485744380 x1862022114772992/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767810 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767785.483101:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744380 x1862022114772992/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767810 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767785.483106:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767785.483107:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767785.483109:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114772992:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767785.483129:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114773120:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775767785.483139:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767785.483145:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767785/real 1775767785]  req@ffff9f1485746a00 x1862022114773120/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767810 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767785.483156:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746a00 x1862022114773120/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767810 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767785.483164:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767785.483168:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767785.483171:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114773120:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775767785.483177:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114773248:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775767785.483229:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767785.483242:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767785/real 1775767785]  req@ffff9f1485744a80 x1862022114773248/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767810 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767785.483251:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744a80 x1862022114773248/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767810 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767785.483256:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767785.483258:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767785.483260:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114773248:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775767785.483264:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114773376:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775767785.483278:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767785.483287:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767785/real 1775767785]  req@ffff9f1485747480 x1862022114773376/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767810 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767785.483299:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747480 x1862022114773376/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767810 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767785.483307:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767785.483310:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767785.483313:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114773376:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767785.483319:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114773504:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775767785.483338:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767785.483348:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767785/real 1775767785]  req@ffff9f1485745880 x1862022114773504/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767810 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767785.483359:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745880 x1862022114773504/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767810 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767785.483374:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767785.483377:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767785.483381:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114773504:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767785.483387:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114773632:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775767785.483406:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767785.483415:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767785/real 1775767785]  req@ffff9f1485746300 x1862022114773632/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767810 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767785.483423:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746300 x1862022114773632/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767810 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767785.483428:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767785.483430:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767785.483432:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114773632:192.168.202.110@tcp:8:kworker.0
00010000:00010000:1.0:1775767785.738747:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767786.506649:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767786.762734:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767787.530654:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:0.0:1775767787.786688:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767788.554664:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767788.810681:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767789.578655:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767789.834785:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767790.602648:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:0.0:1775767790.602684:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767790.602690:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1775767790.602694:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767790.602702:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5168
00000100:00080000:0.0:1775767790.602711:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767790.602716:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775767790.602751:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3100 x1862022114773760/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767790.602785:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767790.602789:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775767790.602791:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767790.602796:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5168
00000100:00080000:0.0:1775767790.602801:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767790.602806:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775767790.602814:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114773760:192.168.202.110@tcp:250:kworker.0
00000008:00100000:0.0:1775767790.602818:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767790.602832:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d1180 x1862022114773888/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767790.602840:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767790.602843:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:0.0:1775767790.602845:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767790.602848:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5168
00000100:00080000:0.0:1775767790.602851:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767790.602862:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767790.602865:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767790.602872:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3480 x1862022114774016/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767790.602880:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767790.602882:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775767790.602883:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767790.602886:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5168
00000800:00000100:2.0:1775767790.602890:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:0.0:1775767790.602891:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767790.602896:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767790.602900:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775767790.602909:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0380 x1862022114774144/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767790.602917:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767790.602920:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1775767790.602922:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767790.602926:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5168
00000100:00080000:0.0:1775767790.602930:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00100000:2.0:1775767790.602936:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767790/real 1775767790]  req@ffff9f14a04d3100 x1862022114773760/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767815 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767790.602937:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767790.602946:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775767790.602955:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3100 x1862022114773760/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767815 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767790.602961:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d1f80 x1862022114774272/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767790.602993:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767790.602994:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767790.602999:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775767790.603000:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767790.603001:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:0.0:1775767790.603007:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5168
00000100:00100000:2.0:1775767790.603018:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114773760:192.168.202.110@tcp:250:kworker.0
00000100:00080000:0.0:1775767790.603021:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767790.603026:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767790.603030:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767790.603037:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3b80 x1862022114774400/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775767790.603040:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114773888:192.168.202.110@tcp:38:kworker.0
00000100:00080000:0.0:1775767790.603053:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767790.603055:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:0.0:1775767790.603056:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767790.603059:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5168
00000100:00080000:0.0:1775767790.603063:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767790.603065:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767790.603068:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767790.603076:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3100 x1862022114774528/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:2.0:1775767790.603085:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:0.0:1775767790.603094:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767790.603097:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775767790.603099:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767790.603103:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5168
00000100:00080000:0.0:1775767790.603106:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767790.603110:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775767790.603111:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767790/real 1775767790]  req@ffff9f14a04d1180 x1862022114773888/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767815 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:0.0:1775767790.603114:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775767790.603125:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0e00 x1862022114774656/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775767790.603138:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1180 x1862022114773888/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767815 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767790.603165:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767790.603169:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767790.603172:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114773888:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767790.603177:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114774016:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775767790.603194:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767790.603201:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767790/real 1775767790]  req@ffff9f14a04d3480 x1862022114774016/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767815 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767790.603209:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3480 x1862022114774016/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767815 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767790.603214:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767790.603217:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767790.603218:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114774016:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767790.603223:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114774144:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775767790.603242:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767790.603251:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767790/real 1775767790]  req@ffff9f14a04d0380 x1862022114774144/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767815 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767790.603261:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0380 x1862022114774144/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767815 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767790.603268:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767790.603271:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767790.603273:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114774144:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775767790.603276:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114774272:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775767790.603287:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767790.603292:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767790/real 1775767790]  req@ffff9f14a04d1f80 x1862022114774272/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767815 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767790.603301:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1f80 x1862022114774272/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767815 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767790.603307:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767790.603309:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767790.603311:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114774272:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775767790.603325:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114774400:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775767790.603336:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767790.603342:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767790/real 1775767790]  req@ffff9f14a04d3b80 x1862022114774400/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767815 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767790.603349:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3b80 x1862022114774400/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767815 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767790.603354:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767790.603356:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767790.603358:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114774400:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767790.603361:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114774528:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775767790.603371:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767790.603376:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767790/real 1775767790]  req@ffff9f14a04d3100 x1862022114774528/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767815 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767790.603383:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3100 x1862022114774528/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767815 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767790.603388:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767790.603389:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767790.603391:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114774528:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767790.603394:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114774656:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775767790.603411:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767790.603416:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767790/real 1775767790]  req@ffff9f14a04d0e00 x1862022114774656/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767815 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767790.603422:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0e00 x1862022114774656/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767815 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767790.603431:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767790.603433:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767790.603435:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114774656:192.168.202.110@tcp:8:kworker.0
00010000:00010000:3.0:1775767790.858709:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767791.626704:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767791.882711:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767792.650660:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775767792.906676:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767793.674646:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767793.930715:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767794.698628:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:0.0:1775767794.954657:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767795.722686:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:2.0:1775767795.722693:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767795.722700:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1775767795.722705:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767795.722718:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5173
00000100:00080000:2.0:1775767795.722730:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767795.722738:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775767795.722779:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2d80 x1862022114774784/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767795.722808:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767795.722810:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775767795.722812:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767795.722815:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5173
00000100:00080000:2.0:1775767795.722819:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767795.722822:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767795.722832:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767795.722837:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114774784:192.168.202.110@tcp:250:kworker.0
00000100:00100000:2.0:1775767795.722843:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1180 x1862022114774912/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767795.722852:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767795.722855:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775767795.722856:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767795.722859:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5173
00000100:00080000:2.0:1775767795.722862:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767795.722864:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767795.722867:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775767795.722874:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0000 x1862022114775040/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767795.722880:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767795.722883:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775767795.722884:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767795.722886:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5173
00000100:00080000:2.0:1775767795.722888:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767795.722891:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767795.722894:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775767795.722900:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3b80 x1862022114775168/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767795.722906:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767795.722908:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1775767795.722909:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767795.722913:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5173
00000100:00080000:2.0:1775767795.722917:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767795.722922:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767795.722927:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775767795.722935:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114774912:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767795.722938:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0e00 x1862022114775296/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767795.722951:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767795.722953:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775767795.722955:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767795.722960:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5173
00000100:00080000:2.0:1775767795.722966:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767795.722971:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767795.722975:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767795.722977:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114775040:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767795.722986:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1880 x1862022114775424/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767795.722992:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767795.722995:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775767795.722996:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775767795.723002:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114775168:192.168.202.110@tcp:8:kworker.0
00000100:00080000:2.0:1775767795.723003:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5173
00000100:00080000:2.0:1775767795.723007:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767795.723011:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767795.723014:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775767795.723026:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3100 x1862022114775552/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775767795.723029:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114775296:192.168.202.110@tcp:8:kworker.0
00000100:00080000:2.0:1775767795.723041:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767795.723044:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775767795.723046:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767795.723050:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5173
00000100:00080000:2.0:1775767795.723053:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767795.723055:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775767795.723059:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114775424:192.168.202.110@tcp:38:kworker.0
00000008:00100000:2.0:1775767795.723059:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775767795.723073:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2300 x1862022114775680/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775767795.723228:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114775552:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767795.723262:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114775680:192.168.202.110@tcp:8:kworker.0
00010000:00010000:3.0:1775767795.978738:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767796.746637:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767797.002755:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767797.770723:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767798.026824:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767798.794639:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000400:00000100:0.0:1775767798.794930:0:2390:0:(lib-socket.c:454:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.202.110:988
00000400:00000100:0.0:1775767798.794995:0:2390:0:(acceptor.c:111:lnet_connect_console_error()) Connection to 192.168.202.110@tcp at host 192.168.202.110 was unreachable: the network or that node may be down, or Lustre may be misconfigured.
00000800:00000100:0.0:1775767798.795019:0:2390:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:0.0:1775767798.795079:0:2390:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:0.0:1775767798.795091:0:2390:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:0.0:1775767798.795100:0:2390:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:0.0:1775767798.795110:0:2390:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:0.0:1775767798.795121:0:2390:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:0.0:1775767798.795128:0:2390:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:0.0:1775767798.795137:0:2390:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000100:00100000:2.0:1775767798.795417:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767795/real 1775767798]  req@ffff9f14852d3100 x1862022114775552/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767820 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767798.795452:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3100 x1862022114775552/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767820 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767798.795467:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767798.795474:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767798.795480:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114775552:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767798.795490:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767795/real 1775767798]  req@ffff9f14852d2300 x1862022114775680/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767820 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767798.795506:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2300 x1862022114775680/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767820 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767798.795514:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767798.795518:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767798.795521:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114775680:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775767798.795529:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767795/real 1775767798]  req@ffff9f14852d0e00 x1862022114775296/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767820 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767798.795542:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0e00 x1862022114775296/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767820 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767798.795550:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767798.795589:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767798.795593:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114775296:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775767798.795600:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767795/real 1775767798]  req@ffff9f14852d1880 x1862022114775424/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767820 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767798.795612:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1880 x1862022114775424/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767820 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767798.795620:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767798.795623:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767798.795626:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114775424:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767798.795633:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767795/real 1775767798]  req@ffff9f14852d1180 x1862022114774912/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767820 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767798.795644:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1180 x1862022114774912/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767820 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767798.795652:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767798.795655:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767798.795658:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114774912:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767798.795665:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767795/real 1775767798]  req@ffff9f14852d0000 x1862022114775040/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767820 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767798.795677:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0000 x1862022114775040/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767820 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767798.795686:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767798.795690:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767798.795693:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114775040:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767798.795700:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767795/real 1775767798]  req@ffff9f14852d3b80 x1862022114775168/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767820 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767798.795713:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3b80 x1862022114775168/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767820 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767798.795722:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767798.795726:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767798.795729:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114775168:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775767798.795737:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767795/real 1775767798]  req@ffff9f14852d2d80 x1862022114774784/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767820 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767798.795750:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2d80 x1862022114774784/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767820 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767798.795758:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767798.795762:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767798.795765:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114774784:192.168.202.110@tcp:250:kworker.0
00010000:00010000:2.0:1775767799.050765:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767799.818711:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775767800.074789:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767800.842674:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:1.0:1775767800.842749:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767800.842762:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1775767800.842766:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767800.842776:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5178
00000100:00080000:1.0:1775767800.842787:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767800.842795:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775767800.842834:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff800 x1862022114775808/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767800.842861:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767800.842864:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775767800.842865:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767800.842869:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5178
00000100:00080000:1.0:1775767800.842872:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767800.842875:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767800.842888:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767800.842905:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fed80 x1862022114775936/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767800.842919:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767800.842923:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775767800.842925:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767800.842932:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5178
00000100:00100000:2.0:1775767800.842933:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114775808:192.168.202.110@tcp:250:kworker.0
00000100:00080000:1.0:1775767800.842939:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767800.842948:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767800.842956:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767800.842971:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ffb80 x1862022114776064/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767800.842980:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767800.842984:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775767800.842986:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767800.842990:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5178
00000100:00080000:1.0:1775767800.842995:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767800.843000:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767800.843006:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775767800.843017:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fca80 x1862022114776192/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767800.843028:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767800.843032:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1775767800.843033:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767800.843037:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5178
00000100:00080000:1.0:1775767800.843042:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000800:00000100:2.0:1775767800.843063:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:1.0:1775767800.843081:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767800.843088:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775767800.843104:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fc380 x1862022114776320/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767800.843115:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767800.843120:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775767800.843122:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767800.843127:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5178
00000100:00080000:1.0:1775767800.843131:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:2.0:1775767800.843133:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767800/real 1775767800]  req@ffff9f14ab4ff800 x1862022114775808/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767825 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767800.843139:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767800.843150:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775767800.843166:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff800 x1862022114775808/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767825 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767800.843169:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff100 x1862022114776448/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767800.843198:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767800.843199:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767800.843205:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775767800.843209:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767800.843210:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:1.0:1775767800.843218:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5178
00000100:00100000:2.0:1775767800.843221:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114775808:192.168.202.110@tcp:250:kworker.0
00000100:00080000:1.0:1775767800.843225:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767800.843233:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767800.843239:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767800.843251:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdc00 x1862022114776576/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775767800.843262:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114775936:192.168.202.110@tcp:38:kworker.0
00000100:00080000:1.0:1775767800.843263:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767800.843271:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775767800.843272:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767800.843277:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5178
00000100:00080000:1.0:1775767800.843281:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767800.843287:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767800.843293:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:2.0:1775767800.843300:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767800.843309:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fce00 x1862022114776704/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775767800.843320:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767800/real 1775767800]  req@ffff9f14ab4fed80 x1862022114775936/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767825 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767800.843337:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fed80 x1862022114775936/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767825 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767800.843348:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767800.843352:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767800.843356:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114775936:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767800.843365:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ffb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114776064:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775767800.843390:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767800.843402:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767800/real 1775767800]  req@ffff9f14ab4ffb80 x1862022114776064/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767825 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767800.843422:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ffb80 x1862022114776064/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767825 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767800.843431:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767800.843434:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767800.843438:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ffb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114776064:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767800.843445:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114776192:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775767800.843470:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767800.843482:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767800/real 1775767800]  req@ffff9f14ab4fca80 x1862022114776192/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767825 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767800.843495:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fca80 x1862022114776192/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767825 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767800.843504:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767800.843507:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767800.843511:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114776192:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775767800.843517:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114776320:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775767800.843538:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767800.843550:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767800/real 1775767800]  req@ffff9f14ab4fc380 x1862022114776320/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767825 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767800.843592:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fc380 x1862022114776320/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767825 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767800.843606:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767800.843609:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767800.843612:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114776320:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775767800.843618:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114776448:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775767800.843641:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767800.843666:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767800/real 1775767800]  req@ffff9f14ab4ff100 x1862022114776448/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767825 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767800.843678:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff100 x1862022114776448/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767825 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767800.843686:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767800.843690:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767800.843693:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114776448:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767800.843714:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114776576:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775767800.843736:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767800.843745:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767800/real 1775767800]  req@ffff9f14ab4fdc00 x1862022114776576/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767825 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767800.843757:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdc00 x1862022114776576/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767825 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767800.843764:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767800.843767:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767800.843771:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114776576:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767800.843777:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114776704:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775767800.843797:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767800.843807:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767800/real 1775767800]  req@ffff9f14ab4fce00 x1862022114776704/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767825 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767800.843821:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fce00 x1862022114776704/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767825 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767800.843828:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767800.843831:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767800.843834:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114776704:192.168.202.110@tcp:8:kworker.0
00010000:00010000:3.0:1775767801.098757:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767801.866674:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775767802.122759:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767802.890638:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767803.146721:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767803.914663:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767804.170695:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767804.938629:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767805.194773:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1775767805.962707:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767805.962715:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1775767805.962721:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00100000:2.0:1775767805.962722:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:0.0:1775767805.962732:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5183
00000100:00080000:0.0:1775767805.962744:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767805.962752:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775767805.962788:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0000 x1862022114776832/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767805.962811:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767805.962814:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775767805.962816:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767805.962822:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5183
00000100:00080000:0.0:1775767805.962830:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767805.962835:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767805.962845:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775767805.962853:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114776832:192.168.202.110@tcp:250:kworker.0
00000100:00100000:0.0:1775767805.962861:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3800 x1862022114776960/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767805.962872:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767805.962875:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:0.0:1775767805.962878:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767805.962881:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5183
00000100:00080000:0.0:1775767805.962885:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767805.962889:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767805.962896:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767805.962913:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d2a00 x1862022114777088/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767805.962921:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767805.962925:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775767805.962929:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767805.962932:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5183
00000100:00080000:0.0:1775767805.962935:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767805.962937:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767805.962941:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:2.0:1775767805.962946:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767805.962969:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d2300 x1862022114777216/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767805.962977:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767805.962980:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1775767805.962982:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767805.962984:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5183
00000100:00080000:0.0:1775767805.962986:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767805.962988:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767805.962993:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775767805.963017:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0a80 x1862022114777344/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775767805.963024:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767805/real 1775767805]  req@ffff9f14a04d0000 x1862022114776832/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767830 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767805.963032:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767805.963049:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775767805.963051:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767805.963058:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5183
00000100:00080000:0.0:1775767805.963065:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:2.0:1775767805.963070:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0000 x1862022114776832/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767830 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767805.963077:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00080000:2.0:1775767805.963091:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000008:00100000:0.0:1775767805.963092:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00080000:2.0:1775767805.963106:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767805.963113:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114776832:192.168.202.110@tcp:250:kworker.0
00000100:00100000:0.0:1775767805.963114:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d1500 x1862022114777472/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767805.963135:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767805.963138:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:0.0:1775767805.963139:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767805.963144:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5183
00000100:00080000:0.0:1775767805.963146:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767805.963150:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767805.963154:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775767805.963154:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114776960:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767805.963182:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0000 x1862022114777600/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767805.963195:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000800:00000100:2.0:1775767805.963199:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:0.0:1775767805.963208:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775767805.963211:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767805.963218:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5183
00000100:00100000:2.0:1775767805.963223:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767805/real 1775767805]  req@ffff9f14a04d3800 x1862022114776960/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767830 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767805.963224:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767805.963240:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767805.963248:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775767805.963249:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3800 x1862022114776960/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767830 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767805.963268:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00100000:0.0:1775767805.963274:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0e00 x1862022114777728/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767805.963279:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767805.963294:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114776960:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767805.963319:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114777088:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775767805.963340:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767805.963355:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767805/real 1775767805]  req@ffff9f14a04d2a00 x1862022114777088/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767830 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767805.963363:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d2a00 x1862022114777088/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767830 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767805.963369:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767805.963371:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767805.963373:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114777088:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767805.963377:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114777216:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775767805.963388:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767805.963393:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767805/real 1775767805]  req@ffff9f14a04d2300 x1862022114777216/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767830 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767805.963406:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d2300 x1862022114777216/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767830 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767805.963411:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767805.963413:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767805.963414:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114777216:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775767805.963418:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114777344:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775767805.963428:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767805.963433:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767805/real 1775767805]  req@ffff9f14a04d0a80 x1862022114777344/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767830 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767805.963440:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0a80 x1862022114777344/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767830 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767805.963445:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767805.963455:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767805.963457:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114777344:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775767805.963461:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114777472:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775767805.963471:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767805.963478:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767805/real 1775767805]  req@ffff9f14a04d1500 x1862022114777472/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767830 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767805.963485:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1500 x1862022114777472/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767830 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767805.963495:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767805.963497:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767805.963499:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114777472:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767805.963514:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114777600:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775767805.963524:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767805.963530:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767805/real 1775767805]  req@ffff9f14a04d0000 x1862022114777600/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767830 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767805.963537:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0000 x1862022114777600/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767830 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767805.963542:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767805.963544:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767805.963546:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114777600:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767805.963550:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114777728:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775767805.963594:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767805.963606:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767805/real 1775767805]  req@ffff9f14a04d0e00 x1862022114777728/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767830 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767805.963614:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0e00 x1862022114777728/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767830 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767805.963620:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767805.963622:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767805.963625:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114777728:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775767806.218770:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767806.986721:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775767807.242764:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767808.010706:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767808.266738:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767809.034697:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775767809.290829:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767810.058686:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767810.314727:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1775767811.082692:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:2.0:1775767811.082699:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:0.0:1775767811.082709:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1775767811.082714:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767811.082727:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5188
00000100:00080000:0.0:1775767811.082742:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767811.082751:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775767811.082803:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3100 x1862022114777856/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767811.082840:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767811.082852:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775767811.082854:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767811.082859:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5188
00000100:00080000:0.0:1775767811.082864:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767811.082870:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767811.082882:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775767811.082888:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114777856:192.168.202.110@tcp:250:kworker.0
00000100:00100000:0.0:1775767811.082900:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3b80 x1862022114777984/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767811.082914:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767811.082920:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:0.0:1775767811.082922:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767811.082927:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5188
00000100:00080000:0.0:1775767811.082932:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767811.082937:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767811.082943:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000800:00000100:2.0:1775767811.082976:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767811.083000:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d1f80 x1862022114778112/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767811.083014:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767811.083018:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775767811.083020:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767811.083025:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5188
00000100:00080000:0.0:1775767811.083031:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00100000:2.0:1775767811.083034:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767811/real 1775767811]  req@ffff9f14a04d3100 x1862022114777856/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767836 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767811.083039:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767811.083051:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775767811.083063:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3100 x1862022114777856/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767836 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767811.083069:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0380 x1862022114778240/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767811.083077:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767811.083086:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:0.0:1775767811.083087:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:2.0:1775767811.083091:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114777856:192.168.202.110@tcp:250:kworker.0
00000100:00080000:0.0:1775767811.083094:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1775767811.083098:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767811.083104:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5188
00000100:00080000:0.0:1775767811.083108:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767811.083113:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767811.083119:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775767811.083120:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114777984:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767811.083137:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3480 x1862022114778368/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:2.0:1775767811.083148:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:0.0:1775767811.083152:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767811.083157:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775767811.083159:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:2.0:1775767811.083161:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767811/real 1775767811]  req@ffff9f14a04d3b80 x1862022114777984/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767836 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767811.083168:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5188
00000100:00080000:0.0:1775767811.083176:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:2.0:1775767811.083184:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3b80 x1862022114777984/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767836 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767811.083185:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00080000:2.0:1775767811.083197:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000008:00100000:0.0:1775767811.083201:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00080000:2.0:1775767811.083204:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767811.083209:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114777984:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767811.083218:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114778112:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767811.083223:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d1180 x1862022114778496/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767811.083235:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767811.083239:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:0.0:1775767811.083241:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767811.083249:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5188
00000800:00000100:2.0:1775767811.083252:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:0.0:1775767811.083257:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767811.083264:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775767811.083267:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767811/real 1775767811]  req@ffff9f14a04d1f80 x1862022114778112/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767836 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767811.083277:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1f80 x1862022114778112/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767836 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:0.0:1775767811.083278:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00080000:2.0:1775767811.083290:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767811.083293:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767811.083298:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3100 x1862022114778624/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775767811.083299:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114778112:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767811.083310:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114778240:192.168.202.110@tcp:8:kworker.0
00000100:00080000:0.0:1775767811.083321:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767811.083326:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775767811.083329:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000800:00000100:2.0:1775767811.083333:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:0.0:1775767811.083340:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5188
00000100:00100000:2.0:1775767811.083344:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767811/real 1775767811]  req@ffff9f14a04d0380 x1862022114778240/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767836 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767811.083347:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767811.083357:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775767811.083359:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0380 x1862022114778240/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767836 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:0.0:1775767811.083368:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00080000:2.0:1775767811.083369:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767811.083375:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767811.083378:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114778240:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775767811.083388:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0e00 x1862022114778752/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775767811.083394:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114778368:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775767811.083409:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767811.083421:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767811/real 1775767811]  req@ffff9f14a04d3480 x1862022114778368/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767836 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767811.083434:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3480 x1862022114778368/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767836 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767811.083442:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767811.083446:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767811.083449:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114778368:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775767811.083455:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114778496:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775767811.083476:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767811.083487:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767811/real 1775767811]  req@ffff9f14a04d1180 x1862022114778496/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767836 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767811.083499:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1180 x1862022114778496/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767836 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767811.083505:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767811.083507:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767811.083509:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114778496:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767811.083513:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114778624:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775767811.083528:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767811.083535:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767811/real 1775767811]  req@ffff9f14a04d3100 x1862022114778624/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767836 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767811.083547:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3100 x1862022114778624/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767836 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767811.083553:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767811.083573:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767811.083575:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114778624:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767811.083591:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114778752:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775767811.083605:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767811.083612:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767811/real 1775767811]  req@ffff9f14a04d0e00 x1862022114778752/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767836 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767811.083620:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0e00 x1862022114778752/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767836 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767811.083625:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767811.083628:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767811.083630:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114778752:192.168.202.110@tcp:8:kworker.0
00010000:00010000:3.0:1775767811.338778:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767812.106691:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767812.362777:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767813.130696:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767813.386747:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767814.154677:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775767814.410734:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767815.178686:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767815.434784:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1775767816.202701:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1775767816.202708:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:1.0:1775767816.202711:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1775767816.202718:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767816.202730:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5193
00000100:00080000:1.0:1775767816.202741:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767816.202748:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775767816.202804:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fea00 x1862022114778880/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767816.202831:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767816.202834:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775767816.202836:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767816.202841:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5193
00000100:00080000:1.0:1775767816.202845:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767816.202849:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767816.202867:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767816.202884:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fd880 x1862022114779008/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775767816.202913:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114778880:192.168.202.110@tcp:250:kworker.0
00000100:00080000:1.0:1775767816.202916:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767816.202919:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775767816.202921:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767816.202926:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5193
00000100:00080000:1.0:1775767816.202932:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767816.202937:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767816.202941:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767816.202952:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fe300 x1862022114779136/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767816.202958:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767816.202960:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775767816.202961:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767816.202964:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5193
00000100:00080000:1.0:1775767816.202966:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767816.202969:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767816.202973:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775767816.202979:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdf80 x1862022114779264/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767816.202985:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767816.202989:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1775767816.202991:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767816.202992:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5193
00000100:00080000:1.0:1775767816.202995:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767816.202996:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767816.202999:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775767816.203005:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fce00 x1862022114779392/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767816.203012:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767816.203015:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775767816.203016:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767816.203020:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5193
00000100:00080000:1.0:1775767816.203023:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767816.203028:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000800:00000100:2.0:1775767816.203034:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000008:00100000:1.0:1775767816.203038:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767816.203054:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdc00 x1862022114779520/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767816.203077:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767816.203080:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775767816.203081:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767816.203087:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5193
00000100:00080000:1.0:1775767816.203091:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767816.203095:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767816.203101:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775767816.203106:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767816/real 1775767816]  req@ffff9f14ab4fea00 x1862022114778880/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767841 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767816.203115:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff100 x1862022114779648/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767816.203153:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767816.203171:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775767816.203174:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767816.203180:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5193
00000100:00080000:1.0:1775767816.203186:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:2.0:1775767816.203191:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fea00 x1862022114778880/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767841 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767816.203195:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767816.203201:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775767816.203212:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fc380 x1862022114779776/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767816.203231:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767816.203243:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767816.203250:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114778880:192.168.202.110@tcp:250:kworker.0
00000100:00100000:2.0:1775767816.203282:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114779008:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775767816.203312:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767816.203334:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767816/real 1775767816]  req@ffff9f14ab4fd880 x1862022114779008/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767841 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767816.203349:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fd880 x1862022114779008/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767841 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767816.203359:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767816.203363:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767816.203366:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114779008:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767816.203374:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114779136:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775767816.203398:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767816.203411:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767816/real 1775767816]  req@ffff9f14ab4fe300 x1862022114779136/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767841 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767816.203426:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fe300 x1862022114779136/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767841 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767816.203442:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767816.203446:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767816.203451:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114779136:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767816.203458:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114779264:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775767816.203481:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767816.203493:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767816/real 1775767816]  req@ffff9f14ab4fdf80 x1862022114779264/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767841 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767816.203507:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdf80 x1862022114779264/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767841 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767816.203517:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767816.203521:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767816.203525:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114779264:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775767816.203533:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114779392:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775767816.203601:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767816.203615:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767816/real 1775767816]  req@ffff9f14ab4fce00 x1862022114779392/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767841 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767816.203629:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fce00 x1862022114779392/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767841 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767816.203639:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767816.203643:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767816.203648:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114779392:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775767816.203655:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114779520:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775767816.203680:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767816.203692:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767816/real 1775767816]  req@ffff9f14ab4fdc00 x1862022114779520/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767841 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767816.203706:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdc00 x1862022114779520/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767841 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767816.203714:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767816.203719:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767816.203722:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114779520:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767816.203730:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114779648:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775767816.203754:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767816.203766:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767816/real 1775767816]  req@ffff9f14ab4ff100 x1862022114779648/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767841 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767816.203779:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff100 x1862022114779648/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767841 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767816.203787:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767816.203791:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767816.203795:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114779648:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767816.203802:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114779776:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775767816.203827:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767816.203840:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767816/real 1775767816]  req@ffff9f14ab4fc380 x1862022114779776/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767841 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767816.203853:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fc380 x1862022114779776/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767841 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767816.203862:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767816.203865:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767816.203869:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114779776:192.168.202.110@tcp:8:kworker.0
00010000:00010000:1.0:1775767816.458827:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767817.226694:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767817.482733:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767818.250634:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767818.506669:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767819.274692:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767819.530728:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767820.298642:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767820.554673:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1775767821.322673:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:2.0:1775767821.322673:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:1.0:1775767821.322683:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1775767821.322688:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767821.322697:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5198
00000100:00080000:1.0:1775767821.322708:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767821.322716:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775767821.322762:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fca80 x1862022114779904/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767821.322789:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767821.322792:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775767821.322794:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767821.322799:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5198
00000100:00080000:1.0:1775767821.322802:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767821.322807:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767821.322817:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775767821.322829:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114779904:192.168.202.110@tcp:250:kworker.0
00000100:00100000:1.0:1775767821.322833:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ffb80 x1862022114780032/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767821.322847:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767821.322865:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775767821.322867:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767821.322872:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5198
00000100:00080000:1.0:1775767821.322876:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767821.322879:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767821.322884:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767821.322896:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fed80 x1862022114780160/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767821.322907:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767821.322910:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775767821.322912:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000800:00000100:2.0:1775767821.322917:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:1.0:1775767821.322918:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5198
00000100:00080000:1.0:1775767821.322924:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767821.322928:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767821.322933:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775767821.322944:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff800 x1862022114780288/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767821.322954:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767821.322958:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1775767821.322960:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767821.322965:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5198
00000100:00100000:2.0:1775767821.322965:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767821/real 1775767821]  req@ffff9f14ab4fca80 x1862022114779904/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767846 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767821.322972:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767821.322979:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767821.322987:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775767821.322997:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fca80 x1862022114779904/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767846 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767821.323001:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fc380 x1862022114780416/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767821.323027:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767821.323028:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767821.323034:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775767821.323037:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767821.323039:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:1.0:1775767821.323044:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5198
00000100:00080000:1.0:1775767821.323049:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:2.0:1775767821.323049:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114779904:192.168.202.110@tcp:250:kworker.0
00000100:00080000:1.0:1775767821.323056:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767821.323062:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767821.323072:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff100 x1862022114780544/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775767821.323080:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ffb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114780032:192.168.202.110@tcp:38:kworker.0
00000100:00080000:1.0:1775767821.323085:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767821.323089:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775767821.323090:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767821.323095:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5198
00000100:00080000:1.0:1775767821.323099:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767821.323103:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767821.323107:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000800:00000100:2.0:1775767821.323112:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767821.323121:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdc00 x1862022114780672/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775767821.323130:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767821/real 1775767821]  req@ffff9f14ab4ffb80 x1862022114780032/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767846 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767821.323157:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767821.323165:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775767821.323169:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767821.323178:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5198
00000100:00080000:1.0:1775767821.323182:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:2.0:1775767821.323185:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ffb80 x1862022114780032/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767846 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767821.323188:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767821.323196:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00080000:2.0:1775767821.323200:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767821.323207:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767821.323209:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fce00 x1862022114780800/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775767821.323215:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ffb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114780032:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767821.323230:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114780160:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775767821.323253:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767821.323264:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767821/real 1775767821]  req@ffff9f14ab4fed80 x1862022114780160/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767846 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767821.323276:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fed80 x1862022114780160/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767846 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767821.323285:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767821.323288:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767821.323292:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114780160:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767821.323298:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114780288:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775767821.323316:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767821.323325:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767821/real 1775767821]  req@ffff9f14ab4ff800 x1862022114780288/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767846 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767821.323337:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff800 x1862022114780288/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767846 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767821.323351:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767821.323355:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767821.323358:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114780288:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775767821.323364:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114780416:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775767821.323383:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767821.323394:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767821/real 1775767821]  req@ffff9f14ab4fc380 x1862022114780416/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767846 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767821.323406:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fc380 x1862022114780416/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767846 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767821.323415:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767821.323418:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767821.323421:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114780416:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775767821.323439:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114780544:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775767821.323460:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767821.323472:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767821/real 1775767821]  req@ffff9f14ab4ff100 x1862022114780544/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767846 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767821.323484:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff100 x1862022114780544/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767846 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767821.323492:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767821.323495:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767821.323499:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114780544:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767821.323505:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114780672:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775767821.323524:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767821.323535:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767821/real 1775767821]  req@ffff9f14ab4fdc00 x1862022114780672/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767846 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767821.323547:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdc00 x1862022114780672/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767846 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767821.323577:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767821.323581:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767821.323585:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114780672:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767821.323591:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114780800:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775767821.323612:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767821.323623:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767821/real 1775767821]  req@ffff9f14ab4fce00 x1862022114780800/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767846 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767821.323635:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fce00 x1862022114780800/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767846 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767821.323643:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767821.323646:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767821.323649:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114780800:192.168.202.110@tcp:8:kworker.0
00010000:00010000:1.0:1775767821.578776:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767822.346643:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767822.602682:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767823.370628:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775767823.626705:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767824.394706:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767824.650785:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767825.418658:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767825.674792:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767826.442642:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:1.0:1775767826.442656:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767826.442664:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1775767826.442669:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767826.442680:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5203
00000100:00080000:1.0:1775767826.442691:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767826.442698:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775767826.442742:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdf80 x1862022114780928/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767826.442771:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767826.442774:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775767826.442775:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767826.442779:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5203
00000100:00080000:1.0:1775767826.442783:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767826.442788:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767826.442812:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775767826.442825:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114780928:192.168.202.110@tcp:250:kworker.0
00000100:00100000:1.0:1775767826.442829:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fe300 x1862022114781056/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767826.442850:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767826.442852:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775767826.442854:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767826.442860:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5203
00000100:00080000:1.0:1775767826.442863:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767826.442866:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767826.442869:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767826.442879:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fd880 x1862022114781184/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767826.442886:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767826.442889:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775767826.442891:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767826.442894:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5203
00000100:00080000:1.0:1775767826.442897:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767826.442899:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767826.442903:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:2.0:1775767826.442910:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767826.442918:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fea00 x1862022114781312/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767826.442931:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767826.442938:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1775767826.442940:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767826.442944:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5203
00000100:00080000:1.0:1775767826.442948:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767826.442952:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767826.442957:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775767826.442957:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767826/real 1775767826]  req@ffff9f14ab4fdf80 x1862022114780928/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767851 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767826.442976:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fce00 x1862022114781440/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775767826.442984:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdf80 x1862022114780928/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767851 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767826.442999:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767826.443009:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775767826.443012:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767826.443013:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767826.443019:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5203
00000100:00080000:1.0:1775767826.443025:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767826.443025:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767826.443035:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114780928:192.168.202.110@tcp:250:kworker.0
00000100:00080000:1.0:1775767826.443036:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767826.443047:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767826.443059:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdc00 x1862022114781568/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767826.443069:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:2.0:1775767826.443073:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114781056:192.168.202.110@tcp:38:kworker.0
00000100:00080000:1.0:1775767826.443081:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775767826.443084:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767826.443089:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5203
00000100:00080000:1.0:1775767826.443096:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767826.443100:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767826.443104:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000800:00000100:2.0:1775767826.443110:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767826.443128:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff100 x1862022114781696/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775767826.443134:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767826/real 1775767826]  req@ffff9f14ab4fe300 x1862022114781056/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767851 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767826.443172:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767826.443182:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775767826.443184:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:2.0:1775767826.443186:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fe300 x1862022114781056/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767851 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767826.443192:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5203
00000100:00080000:1.0:1775767826.443200:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767826.443213:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00080000:2.0:1775767826.443215:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000008:00100000:1.0:1775767826.443222:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00080000:2.0:1775767826.443228:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767826.443233:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114781056:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775767826.443239:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fc380 x1862022114781824/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775767826.443251:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114781184:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775767826.443282:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767826.443296:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767826/real 1775767826]  req@ffff9f14ab4fd880 x1862022114781184/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767851 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767826.443309:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fd880 x1862022114781184/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767851 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767826.443317:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767826.443320:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767826.443322:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114781184:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767826.443327:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114781312:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775767826.443344:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767826.443352:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767826/real 1775767826]  req@ffff9f14ab4fea00 x1862022114781312/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767851 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767826.443359:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fea00 x1862022114781312/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767851 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767826.443365:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767826.443367:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767826.443369:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114781312:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775767826.443372:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114781440:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775767826.443382:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767826.443388:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767826/real 1775767826]  req@ffff9f14ab4fce00 x1862022114781440/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767851 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767826.443394:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fce00 x1862022114781440/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767851 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767826.443400:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767826.443402:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767826.443404:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114781440:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775767826.443417:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114781568:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775767826.443427:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767826.443433:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767826/real 1775767826]  req@ffff9f14ab4fdc00 x1862022114781568/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767851 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767826.443440:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdc00 x1862022114781568/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767851 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767826.443445:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767826.443447:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767826.443449:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114781568:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767826.443453:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114781696:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775767826.443463:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767826.443477:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767826/real 1775767826]  req@ffff9f14ab4ff100 x1862022114781696/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767851 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767826.443485:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff100 x1862022114781696/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767851 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767826.443490:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767826.443492:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767826.443495:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114781696:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767826.443499:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114781824:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775767826.443510:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767826.443516:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767826/real 1775767826]  req@ffff9f14ab4fc380 x1862022114781824/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767851 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767826.443523:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fc380 x1862022114781824/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767851 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767826.443528:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767826.443530:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767826.443532:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114781824:192.168.202.110@tcp:8:kworker.0
00010000:00010000:3.0:1775767826.698745:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767827.466662:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767827.722709:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767828.490658:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775767828.746766:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767829.514655:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767829.770720:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767830.538717:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775767830.794771:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767831.562615:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:0.0:1775767831.562743:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767831.562777:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1775767831.562782:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767831.562792:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5209
00000100:00080000:0.0:1775767831.562804:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767831.562811:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775767831.562848:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0000 x1862022114781952/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767831.562872:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767831.562875:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775767831.562876:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767831.562879:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5209
00000100:00080000:0.0:1775767831.562883:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767831.562886:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767831.562894:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767831.562905:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d1500 x1862022114782080/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775767831.562905:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114781952:192.168.202.110@tcp:250:kworker.0
00000100:00080000:0.0:1775767831.562925:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767831.562927:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:0.0:1775767831.562929:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767831.562932:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5209
00000100:00080000:0.0:1775767831.562934:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767831.562937:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767831.562940:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767831.562947:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0a80 x1862022114782208/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767831.562954:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767831.562957:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775767831.562958:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767831.562960:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5209
00000100:00080000:0.0:1775767831.562962:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767831.562966:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000800:00000100:2.0:1775767831.562967:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000008:00100000:0.0:1775767831.562971:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775767831.562980:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d2300 x1862022114782336/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767831.562990:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767831.562992:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1775767831.562994:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767831.562996:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5209
00000100:00080000:0.0:1775767831.562999:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767831.563002:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767831.563005:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775767831.563006:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767831/real 1775767831]  req@ffff9f14a04d0000 x1862022114781952/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767856 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767831.563015:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d2a00 x1862022114782464/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767831.563022:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:2.0:1775767831.563025:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0000 x1862022114781952/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767856 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767831.563026:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775767831.563030:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767831.563038:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5209
00000100:00080000:2.0:1775767831.563041:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767831.563042:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767831.563048:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00080000:2.0:1775767831.563050:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000008:00100000:0.0:1775767831.563054:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775767831.563056:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114781952:192.168.202.110@tcp:250:kworker.0
00000100:00100000:0.0:1775767831.563064:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3800 x1862022114782592/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767831.563076:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767831.563080:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00100000:2.0:1775767831.563080:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114782080:192.168.202.110@tcp:38:kworker.0
00000100:00080000:0.0:1775767831.563083:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767831.563091:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5209
00000100:00080000:0.0:1775767831.563093:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767831.563096:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767831.563100:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767831.563108:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0000 x1862022114782720/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:2.0:1775767831.563117:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:0.0:1775767831.563118:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767831.563122:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775767831.563124:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767831.563127:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5209
00000100:00080000:0.0:1775767831.563130:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767831.563133:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767831.563136:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775767831.563138:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767831/real 1775767831]  req@ffff9f14a04d1500 x1862022114782080/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767856 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767831.563146:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0e00 x1862022114782848/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775767831.563157:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1500 x1862022114782080/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767856 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767831.563164:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767831.563175:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767831.563179:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114782080:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767831.563184:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114782208:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775767831.563200:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767831.563207:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767831/real 1775767831]  req@ffff9f14a04d0a80 x1862022114782208/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767856 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767831.563215:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0a80 x1862022114782208/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767856 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767831.563220:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767831.563222:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767831.563224:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114782208:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767831.563227:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114782336:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775767831.563238:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767831.563249:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767831/real 1775767831]  req@ffff9f14a04d2300 x1862022114782336/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767856 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767831.563256:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d2300 x1862022114782336/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767856 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767831.563261:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767831.563263:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767831.563264:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114782336:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775767831.563268:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114782464:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775767831.563278:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767831.563283:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767831/real 1775767831]  req@ffff9f14a04d2a00 x1862022114782464/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767856 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767831.563290:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d2a00 x1862022114782464/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767856 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767831.563295:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767831.563296:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767831.563298:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114782464:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775767831.563311:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114782592:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775767831.563320:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767831.563326:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767831/real 1775767831]  req@ffff9f14a04d3800 x1862022114782592/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767856 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767831.563338:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3800 x1862022114782592/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767856 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767831.563343:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767831.563345:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767831.563347:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114782592:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767831.563350:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114782720:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775767831.563359:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767831.563365:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767831/real 1775767831]  req@ffff9f14a04d0000 x1862022114782720/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767856 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767831.563372:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0000 x1862022114782720/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767856 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767831.563377:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767831.563378:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767831.563380:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114782720:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767831.563383:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114782848:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775767831.563391:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767831.563396:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767831/real 1775767831]  req@ffff9f14a04d0e00 x1862022114782848/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767856 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767831.563402:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0e00 x1862022114782848/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767856 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767831.563411:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767831.563413:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767831.563415:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114782848:192.168.202.110@tcp:8:kworker.0
00010000:00010000:3.0:1775767831.818679:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767832.586627:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767832.842762:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767833.610630:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767833.866712:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767834.634693:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767834.890688:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767835.658665:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775767835.914694:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1775767836.682779:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767836.682789:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1775767836.682794:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00100000:2.0:1775767836.682798:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:0.0:1775767836.682806:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5214
00000100:00080000:0.0:1775767836.682820:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767836.682827:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775767836.682870:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3100 x1862022114782976/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767836.682900:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767836.682903:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775767836.682905:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767836.682908:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5214
00000100:00080000:0.0:1775767836.682911:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767836.682914:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767836.682923:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767836.682935:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d1180 x1862022114783104/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775767836.682937:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114782976:192.168.202.110@tcp:250:kworker.0
00000100:00080000:0.0:1775767836.682962:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767836.682964:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:0.0:1775767836.682965:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767836.682971:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5214
00000100:00080000:0.0:1775767836.682973:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767836.682976:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767836.682979:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767836.682987:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3480 x1862022114783232/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767836.682993:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767836.682996:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775767836.682997:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767836.683000:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5214
00000100:00080000:0.0:1775767836.683003:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767836.683007:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767836.683012:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775767836.683020:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0380 x1862022114783360/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:2.0:1775767836.683025:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:0.0:1775767836.683036:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767836.683040:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1775767836.683042:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767836.683046:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5214
00000100:00080000:0.0:1775767836.683053:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767836.683056:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767836.683060:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775767836.683071:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d1f80 x1862022114783488/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775767836.683081:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767836/real 1775767836]  req@ffff9f14a04d3100 x1862022114782976/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767861 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767836.683083:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767836.683096:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775767836.683101:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767836.683114:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5214
00000100:00080000:0.0:1775767836.683124:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767836.683128:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767836.683132:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775767836.683135:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3100 x1862022114782976/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767861 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767836.683146:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3b80 x1862022114783616/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767836.683148:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767836.683164:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767836.683182:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114782976:192.168.202.110@tcp:250:kworker.0
00000100:00080000:0.0:1775767836.683186:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767836.683190:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:0.0:1775767836.683192:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767836.683196:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5214
00000100:00080000:0.0:1775767836.683200:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767836.683203:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767836.683208:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775767836.683213:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114783104:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767836.683220:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3100 x1862022114783744/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767836.683230:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767836.683233:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775767836.683235:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000800:00000100:2.0:1775767836.683239:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:0.0:1775767836.683246:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5214
00000100:00080000:0.0:1775767836.683251:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767836.683258:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775767836.683258:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767836/real 1775767836]  req@ffff9f14a04d1180 x1862022114783104/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767861 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:0.0:1775767836.683270:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775767836.683283:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1180 x1862022114783104/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767861 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767836.683284:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0e00 x1862022114783872/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767836.683336:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767836.683343:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767836.683347:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114783104:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767836.683355:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114783232:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775767836.683380:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767836.683392:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767836/real 1775767836]  req@ffff9f14a04d3480 x1862022114783232/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767861 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767836.683405:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3480 x1862022114783232/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767861 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767836.683414:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767836.683417:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767836.683419:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114783232:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767836.683424:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114783360:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775767836.683441:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767836.683448:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767836/real 1775767836]  req@ffff9f14a04d0380 x1862022114783360/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767861 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767836.683456:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0380 x1862022114783360/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767861 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767836.683461:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767836.683463:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767836.683465:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114783360:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775767836.683470:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114783488:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775767836.683480:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767836.683485:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767836/real 1775767836]  req@ffff9f14a04d1f80 x1862022114783488/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767861 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767836.683492:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1f80 x1862022114783488/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767861 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767836.683497:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767836.683499:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767836.683501:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114783488:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775767836.683504:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114783616:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775767836.683514:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767836.683519:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767836/real 1775767836]  req@ffff9f14a04d3b80 x1862022114783616/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767861 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767836.683526:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3b80 x1862022114783616/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767861 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767836.683531:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767836.683533:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767836.683535:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114783616:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767836.683550:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114783744:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775767836.683617:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767836.683629:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767836/real 1775767836]  req@ffff9f14a04d3100 x1862022114783744/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767861 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767836.683642:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3100 x1862022114783744/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767861 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767836.683651:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767836.683654:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767836.683657:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114783744:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767836.683664:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114783872:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775767836.683686:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767836.683698:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767836/real 1775767836]  req@ffff9f14a04d0e00 x1862022114783872/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767861 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767836.683727:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0e00 x1862022114783872/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767861 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767836.683736:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767836.683740:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767836.683743:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114783872:192.168.202.110@tcp:8:kworker.0
00010000:00010000:3.0:1775767836.938881:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767837.706619:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775767837.962740:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767838.730663:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767838.986721:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767839.754747:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767840.010749:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767840.778644:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767841.034784:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767841.802767:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:2.0:1775767841.802813:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767841.802824:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1775767841.802829:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767841.802842:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5219
00000100:00080000:2.0:1775767841.802856:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767841.802866:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775767841.802907:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2d80 x1862022114784000/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767841.802937:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767841.802941:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775767841.802943:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767841.802951:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5219
00000100:00080000:2.0:1775767841.802957:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767841.802962:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767841.802980:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767841.802987:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114784000:192.168.202.110@tcp:250:kworker.0
00000100:00100000:2.0:1775767841.803001:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3b80 x1862022114784128/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767841.803012:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767841.803015:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775767841.803016:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767841.803020:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5219
00000100:00080000:2.0:1775767841.803026:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767841.803029:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767841.803033:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775767841.803040:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0000 x1862022114784256/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767841.803054:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000800:00000100:1.0:1775767841.803062:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:2.0:1775767841.803064:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775767841.803067:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767841.803072:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5219
00000100:00080000:2.0:1775767841.803077:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767841.803081:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767841.803086:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775767841.803098:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1180 x1862022114784384/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767841.803105:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767841.803108:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1775767841.803110:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:1.0:1775767841.803115:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767841/real 1775767841]  req@ffff9f14852d2d80 x1862022114784000/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767866 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767841.803116:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5219
00000100:00080000:2.0:1775767841.803122:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767841.803130:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767841.803138:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775767841.803151:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1880 x1862022114784512/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767841.803159:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767841.803161:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775767841.803166:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:1.0:1775767841.803167:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2d80 x1862022114784000/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767866 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767841.803178:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5219
00000100:00080000:2.0:1775767841.803189:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767841.803201:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767841.803214:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00080000:1.0:1775767841.803224:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00100000:2.0:1775767841.803228:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0e00 x1862022114784640/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767841.803234:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767841.803250:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114784000:192.168.202.110@tcp:250:kworker.0
00000100:00080000:2.0:1775767841.803264:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767841.803266:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775767841.803268:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767841.803274:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5219
00000100:00080000:2.0:1775767841.803279:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767841.803281:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767841.803288:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767841.803294:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114784128:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767841.803309:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2300 x1862022114784768/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767841.803319:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767841.803322:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000800:00000100:1.0:1775767841.803329:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:2.0:1775767841.803329:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767841.803343:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5219
00000100:00080000:2.0:1775767841.803352:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767841.803358:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775767841.803362:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767841/real 1775767841]  req@ffff9f14852d3b80 x1862022114784128/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767866 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:2.0:1775767841.803366:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775767841.803383:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3b80 x1862022114784128/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767866 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767841.803384:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3100 x1862022114784896/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767841.803457:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767841.803472:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767841.803479:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114784128:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775767841.803485:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114784256:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775767841.803504:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767841.803512:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767841/real 1775767841]  req@ffff9f14852d0000 x1862022114784256/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767866 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767841.803520:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0000 x1862022114784256/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767866 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767841.803526:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767841.803528:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767841.803530:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114784256:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775767841.803535:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114784384:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775767841.803546:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767841.803597:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767841/real 1775767841]  req@ffff9f14852d1180 x1862022114784384/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767866 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767841.803611:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1180 x1862022114784384/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767866 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767841.803619:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767841.803622:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767841.803625:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114784384:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775767841.803633:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114784512:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775767841.803668:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767841.803680:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767841/real 1775767841]  req@ffff9f14852d1880 x1862022114784512/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767866 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767841.803692:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1880 x1862022114784512/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767866 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767841.803700:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767841.803703:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767841.803706:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114784512:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775767841.803713:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114784640:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775767841.803733:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767841.803744:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767841/real 1775767841]  req@ffff9f14852d0e00 x1862022114784640/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767866 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767841.803757:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0e00 x1862022114784640/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767866 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767841.803765:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767841.803768:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767841.803771:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114784640:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775767841.803792:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114784768:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775767841.803812:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767841.803821:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767841/real 1775767841]  req@ffff9f14852d2300 x1862022114784768/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767866 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767841.803829:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2300 x1862022114784768/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767866 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767841.803834:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767841.803837:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767841.803839:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114784768:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775767841.803843:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114784896:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775767841.803854:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767841.803860:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767841/real 1775767841]  req@ffff9f14852d3100 x1862022114784896/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767866 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767841.803867:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3100 x1862022114784896/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767866 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767841.803877:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767841.803880:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767841.803882:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114784896:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775767842.058888:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767842.826652:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775767843.082687:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767843.850703:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767844.106881:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767844.874676:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775767845.130732:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767845.898682:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767846.154728:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1775767846.922647:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767846.922655:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00100000:0.0:1775767846.922658:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:2.0:1775767846.922660:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767846.922674:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5224
00000100:00080000:2.0:1775767846.922684:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767846.922690:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775767846.922727:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1c00 x1862022114785024/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767846.922751:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767846.922754:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775767846.922755:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767846.922759:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5224
00000100:00080000:2.0:1775767846.922761:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767846.922764:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767846.922771:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775767846.922780:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3480 x1862022114785152/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767846.922787:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767846.922790:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775767846.922791:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767846.922794:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5224
00000100:00080000:2.0:1775767846.922796:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00100000:1.0:1775767846.922801:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114785024:192.168.202.110@tcp:250:kworker.0
00000100:00080000:2.0:1775767846.922801:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767846.922804:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775767846.922814:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3100 x1862022114785280/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767846.922820:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767846.922823:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775767846.922825:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767846.922827:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5224
00000100:00080000:2.0:1775767846.922829:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767846.922831:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767846.922834:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775767846.922842:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2300 x1862022114785408/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767846.922848:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767846.922851:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1775767846.922852:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767846.922854:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5224
00000100:00080000:2.0:1775767846.922856:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767846.922858:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767846.922861:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:1.0:1775767846.922866:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767846.922870:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0e00 x1862022114785536/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767846.922876:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767846.922879:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775767846.922880:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767846.922883:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5224
00000100:00080000:2.0:1775767846.922886:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767846.922889:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767846.922892:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775767846.922899:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1880 x1862022114785664/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:1.0:1775767846.922906:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767846/real 1775767846]  req@ffff9f14852d1c00 x1862022114785024/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767871 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767846.922922:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1c00 x1862022114785024/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767871 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767846.922924:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767846.922931:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775767846.922935:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767846.922939:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767846.922939:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5224
00000100:00080000:2.0:1775767846.922945:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767846.922948:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:2.0:1775767846.922951:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775767846.922956:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114785024:192.168.202.110@tcp:250:kworker.0
00000008:00100000:2.0:1775767846.922958:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767846.922964:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114785152:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767846.922969:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1180 x1862022114785792/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767846.922977:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767846.922980:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775767846.922982:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000800:00000100:1.0:1775767846.922987:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:2.0:1775767846.922987:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5224
00000100:00080000:2.0:1775767846.922995:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767846.922998:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767846.923003:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775767846.923006:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767846/real 1775767846]  req@ffff9f14852d3480 x1862022114785152/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767871 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767846.923013:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0000 x1862022114785920/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:1.0:1775767846.923030:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3480 x1862022114785152/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767871 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767846.923037:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767846.923040:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767846.923044:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114785152:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775767846.923066:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114785280:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775767846.923086:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767846.923097:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767846/real 1775767846]  req@ffff9f14852d3100 x1862022114785280/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767871 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767846.923110:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3100 x1862022114785280/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767871 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767846.923119:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767846.923122:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767846.923125:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114785280:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775767846.923129:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114785408:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775767846.923144:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767846.923151:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767846/real 1775767846]  req@ffff9f14852d2300 x1862022114785408/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767871 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767846.923158:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2300 x1862022114785408/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767871 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767846.923164:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767846.923166:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767846.923168:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114785408:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775767846.923171:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114785536:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775767846.923181:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767846.923186:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767846/real 1775767846]  req@ffff9f14852d0e00 x1862022114785536/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767871 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767846.923193:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0e00 x1862022114785536/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767871 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767846.923198:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767846.923199:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767846.923201:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114785536:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775767846.923205:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114785664:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775767846.923213:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767846.923219:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767846/real 1775767846]  req@ffff9f14852d1880 x1862022114785664/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767871 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767846.923226:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1880 x1862022114785664/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767871 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767846.923231:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767846.923232:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767846.923234:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114785664:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775767846.923237:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114785792:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775767846.923246:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767846.923251:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767846/real 1775767846]  req@ffff9f14852d1180 x1862022114785792/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767871 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767846.923262:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1180 x1862022114785792/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767871 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767846.923267:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767846.923268:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767846.923270:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114785792:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775767846.923273:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114785920:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775767846.923295:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767846.923305:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767846/real 1775767846]  req@ffff9f14852d0000 x1862022114785920/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767871 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767846.923317:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0000 x1862022114785920/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767871 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767846.923326:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767846.923330:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767846.923334:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114785920:192.168.202.110@tcp:8:kworker.0
00010000:00010000:3.0:1775767847.178743:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767847.946647:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767848.202775:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767848.970680:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767849.226752:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767849.994682:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775767850.250754:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767851.018673:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767851.274701:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767852.042643:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775767852.042685:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767852.042693:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775767852.042697:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767852.042708:0:8:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5229
00000100:00080000:3.0:1775767852.042720:0:8:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767852.042729:0:8:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775767852.042770:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747b80 x1862022114786048/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767852.042799:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767852.042802:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775767852.042804:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767852.042809:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5229
00000100:00080000:3.0:1775767852.042812:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767852.042816:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767852.042827:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767852.042843:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746300 x1862022114786176/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:1.0:1775767852.042853:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114786048:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775767852.042856:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767852.042862:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775767852.042864:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767852.042868:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5229
00000100:00080000:3.0:1775767852.042872:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767852.042876:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767852.042882:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767852.042895:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745880 x1862022114786304/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767852.042904:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767852.042907:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775767852.042910:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000800:00000100:1.0:1775767852.042914:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775767852.042919:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5229
00000100:00080000:3.0:1775767852.042924:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767852.042929:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767852.042934:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775767852.042948:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747480 x1862022114786432/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:1.0:1775767852.042953:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767852/real 1775767852]  req@ffff9f1485747b80 x1862022114786048/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767877 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767852.042973:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767852.042977:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00100000:1.0:1775767852.042980:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747b80 x1862022114786048/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767877 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767852.042981:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767852.042992:0:8:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5229
00000100:00080000:1.0:1775767852.042994:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767852.042997:0:8:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767852.043000:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:3.0:1775767852.043004:0:8:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775767852.043007:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114786048:192.168.202.110@tcp:250:kworker.0
00000008:00100000:3.0:1775767852.043015:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775767852.043030:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114786176:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767852.043030:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744a80 x1862022114786560/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767852.043048:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767852.043051:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000800:00000100:1.0:1775767852.043055:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775767852.043056:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767852.043063:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5229
00000100:00100000:1.0:1775767852.043068:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767852/real 1775767852]  req@ffff9f1485746300 x1862022114786176/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767877 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767852.043068:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767852.043077:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775767852.043084:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746300 x1862022114786176/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767877 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:3.0:1775767852.043085:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00080000:1.0:1775767852.043097:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767852.043100:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767852.043104:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114786176:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767852.043106:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746a00 x1862022114786688/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:1.0:1775767852.043111:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114786304:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775767852.043127:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767852.043130:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000800:00000100:1.0:1775767852.043133:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775767852.043134:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767852.043141:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5229
00000100:00100000:1.0:1775767852.043145:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767852/real 1775767852]  req@ffff9f1485745880 x1862022114786304/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767877 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767852.043147:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767852.043155:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775767852.043159:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745880 x1862022114786304/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767877 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:3.0:1775767852.043163:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00080000:1.0:1775767852.043166:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767852.043170:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767852.043172:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114786304:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767852.043179:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744380 x1862022114786816/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:1.0:1775767852.043180:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114786432:192.168.202.110@tcp:8:kworker.0
00000100:00080000:3.0:1775767852.043195:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000800:00000100:1.0:1775767852.043198:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775767852.043203:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775767852.043206:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:1.0:1775767852.043209:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767852/real 1775767852]  req@ffff9f1485747480 x1862022114786432/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767877 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767852.043214:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5229
00000100:00080000:3.0:1775767852.043222:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:1.0:1775767852.043227:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747480 x1862022114786432/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767877 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767852.043229:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767852.043237:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775767852.043249:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744e00 x1862022114786944/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767852.043252:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767852.043262:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767852.043269:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114786432:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775767852.043290:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114786560:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775767852.043307:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767852.043315:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767852/real 1775767852]  req@ffff9f1485744a80 x1862022114786560/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767877 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767852.043322:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744a80 x1862022114786560/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767877 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767852.043328:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767852.043331:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767852.043334:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114786560:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775767852.043340:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114786688:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775767852.043357:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767852.043368:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767852/real 1775767852]  req@ffff9f1485746a00 x1862022114786688/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767877 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767852.043381:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746a00 x1862022114786688/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767877 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767852.043390:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767852.043392:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767852.043394:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114786688:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775767852.043398:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114786816:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775767852.043412:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767852.043419:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767852/real 1775767852]  req@ffff9f1485744380 x1862022114786816/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767877 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767852.043427:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744380 x1862022114786816/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767877 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767852.043431:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767852.043434:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767852.043435:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114786816:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775767852.043439:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114786944:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775767852.043448:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767852.043453:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767852/real 1775767852]  req@ffff9f1485744e00 x1862022114786944/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767877 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767852.043460:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744e00 x1862022114786944/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767877 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767852.043469:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767852.043471:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767852.043473:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114786944:192.168.202.110@tcp:8:kworker.0
00010000:00010000:1.0:1775767852.298772:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767853.066698:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767853.322708:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767854.090636:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767854.346710:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767855.114649:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767855.370727:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767856.138635:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767856.394728:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767857.162635:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:1.0:1775767857.162651:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767857.162657:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1775767857.162661:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767857.162669:0:8:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5234
00000100:00080000:1.0:1775767857.162680:0:8:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767857.162686:0:8:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775767857.162723:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff800 x1862022114787072/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767857.162747:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767857.162750:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775767857.162751:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767857.162754:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5234
00000100:00080000:1.0:1775767857.162756:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767857.162759:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767857.162766:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767857.162775:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fed80 x1862022114787200/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767857.162782:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767857.162786:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775767857.162787:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767857.162790:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5234
00000100:00080000:1.0:1775767857.162792:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767857.162794:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767857.162796:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767857.162806:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ffb80 x1862022114787328/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775767857.162807:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114787072:192.168.202.110@tcp:250:kworker.0
00000100:00080000:1.0:1775767857.162819:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767857.162821:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775767857.162822:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767857.162825:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5234
00000100:00080000:1.0:1775767857.162827:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767857.162829:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767857.162832:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775767857.162840:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fca80 x1862022114787456/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767857.162846:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767857.162848:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1775767857.162849:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767857.162852:0:8:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5234
00000100:00080000:1.0:1775767857.162856:0:8:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767857.162858:0:8:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767857.162862:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775767857.162871:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fc380 x1862022114787584/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767857.162881:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000800:00000100:2.0:1775767857.162886:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:1.0:1775767857.162887:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775767857.162891:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767857.162897:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5234
00000100:00080000:1.0:1775767857.162902:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767857.162908:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767857.162914:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767857.162927:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff100 x1862022114787712/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775767857.162934:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767857/real 1775767857]  req@ffff9f14ab4ff800 x1862022114787072/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767882 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767857.162942:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767857.162950:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775767857.162955:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767857.162961:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5234
00000100:00080000:1.0:1775767857.162965:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:2.0:1775767857.162966:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff800 x1862022114787072/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767882 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767857.162975:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00080000:2.0:1775767857.162985:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000008:00100000:1.0:1775767857.162986:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00080000:2.0:1775767857.162996:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767857.163001:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114787072:192.168.202.110@tcp:250:kworker.0
00000100:00100000:1.0:1775767857.163006:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdc00 x1862022114787840/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775767857.163012:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114787200:192.168.202.110@tcp:38:kworker.0
00000100:00080000:1.0:1775767857.163023:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767857.163025:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775767857.163028:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767857.163032:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5234
00000100:00080000:1.0:1775767857.163036:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767857.163040:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000800:00000100:2.0:1775767857.163043:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000008:00100000:1.0:1775767857.163046:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775767857.163057:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767857/real 1775767857]  req@ffff9f14ab4fed80 x1862022114787200/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767882 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767857.163059:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fce00 x1862022114787968/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775767857.163083:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fed80 x1862022114787200/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767882 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767857.163091:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767857.163094:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767857.163097:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114787200:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767857.163118:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ffb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114787328:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775767857.163131:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767857.163138:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767857/real 1775767857]  req@ffff9f14ab4ffb80 x1862022114787328/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767882 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767857.163145:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ffb80 x1862022114787328/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767882 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767857.163151:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767857.163153:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767857.163155:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ffb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114787328:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767857.163158:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114787456:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775767857.163168:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767857.163174:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767857/real 1775767857]  req@ffff9f14ab4fca80 x1862022114787456/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767882 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767857.163180:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fca80 x1862022114787456/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767882 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767857.163186:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767857.163187:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767857.163189:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114787456:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775767857.163193:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114787584:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775767857.163202:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767857.163208:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767857/real 1775767857]  req@ffff9f14ab4fc380 x1862022114787584/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767882 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767857.163215:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fc380 x1862022114787584/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767882 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767857.163219:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767857.163221:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767857.163223:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114787584:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775767857.163226:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114787712:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775767857.163235:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767857.163240:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767857/real 1775767857]  req@ffff9f14ab4ff100 x1862022114787712/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767882 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767857.163247:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff100 x1862022114787712/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767882 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767857.163252:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767857.163253:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767857.163255:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114787712:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767857.163258:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114787840:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775767857.163267:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767857.163271:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767857/real 1775767857]  req@ffff9f14ab4fdc00 x1862022114787840/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767882 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767857.163278:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdc00 x1862022114787840/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767882 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767857.163283:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767857.163285:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767857.163287:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114787840:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767857.163290:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114787968:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775767857.163299:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767857.163304:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767857/real 1775767857]  req@ffff9f14ab4fce00 x1862022114787968/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767882 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767857.163310:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fce00 x1862022114787968/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767882 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767857.163315:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767857.163317:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767857.163319:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114787968:192.168.202.110@tcp:8:kworker.0
00010000:00010000:1.0:1775767857.418694:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767858.186668:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767858.442761:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767859.210616:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775767859.466681:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767860.234648:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767860.490685:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767861.258647:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767861.514667:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1775767862.282636:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1775767862.282637:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:1.0:1775767862.282645:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1775767862.282650:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767862.282658:0:8:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5239
00000100:00080000:1.0:1775767862.282668:0:8:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767862.282674:0:8:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775767862.282710:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fea00 x1862022114788096/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767862.282738:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767862.282742:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775767862.282744:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767862.282749:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5239
00000100:00080000:1.0:1775767862.282753:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767862.282759:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767862.282769:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775767862.282781:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114788096:192.168.202.110@tcp:250:kworker.0
00000100:00100000:1.0:1775767862.282785:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fd880 x1862022114788224/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767862.282795:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767862.282797:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775767862.282798:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767862.282801:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5239
00000100:00080000:1.0:1775767862.282804:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767862.282807:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767862.282810:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767862.282818:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fe300 x1862022114788352/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767862.282824:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767862.282827:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775767862.282828:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767862.282831:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5239
00000100:00080000:1.0:1775767862.282833:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767862.282835:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767862.282838:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775767862.282845:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdf80 x1862022114788480/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767862.282851:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767862.282853:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1775767862.282854:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767862.282857:0:8:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5239
00000100:00080000:1.0:1775767862.282860:0:8:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767862.282863:0:8:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767862.282867:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775767862.282874:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fce00 x1862022114788608/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775767862.282878:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114788224:192.168.202.110@tcp:38:kworker.0
00000100:00080000:1.0:1775767862.282885:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767862.282887:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775767862.282888:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767862.282892:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5239
00000100:00080000:1.0:1775767862.282895:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767862.282898:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767862.282902:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775767862.282910:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114788352:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775767862.282912:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdc00 x1862022114788736/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767862.282921:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767862.282924:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775767862.282926:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:2.0:1775767862.282930:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114788480:192.168.202.110@tcp:8:kworker.0
00000100:00080000:1.0:1775767862.282931:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5239
00000100:00080000:1.0:1775767862.282937:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767862.282940:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767862.282944:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767862.282955:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff100 x1862022114788864/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775767862.282958:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114788608:192.168.202.110@tcp:8:kworker.0
00000100:00080000:1.0:1775767862.282970:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767862.282981:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775767862.282983:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767862.282989:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5239
00000100:00080000:1.0:1775767862.282993:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:2.0:1775767862.282995:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114788736:192.168.202.110@tcp:38:kworker.0
00000100:00080000:1.0:1775767862.282999:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767862.283007:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775767862.283018:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fc380 x1862022114788992/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775767862.283026:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114788864:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767862.283043:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114788992:192.168.202.110@tcp:8:kworker.0
00010000:00010000:3.0:1775767862.538697:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767863.306722:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767863.562675:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767864.330800:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775767864.586744:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767865.354910:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000400:00000100:2.0:1775767865.354927:0:2392:0:(lib-socket.c:454:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.202.110:988
00000400:00000100:2.0:1775767865.354972:0:2392:0:(acceptor.c:111:lnet_connect_console_error()) Connection to 192.168.202.110@tcp at host 192.168.202.110 was unreachable: the network or that node may be down, or Lustre may be misconfigured.
00000800:00000100:2.0:1775767865.354989:0:2392:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:2.0:1775767865.355037:0:2392:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:2.0:1775767865.355047:0:2392:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:2.0:1775767865.355056:0:2392:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:2.0:1775767865.355064:0:2392:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:2.0:1775767865.355071:0:2392:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:2.0:1775767865.355079:0:2392:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:2.0:1775767865.355087:0:2392:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000100:00100000:1.0:1775767865.355164:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767862/real 1775767865]  req@ffff9f14ab4fc380 x1862022114788992/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767887 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767865.355191:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fc380 x1862022114788992/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767887 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767865.355204:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767865.355209:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767865.355214:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114788992:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775767865.355222:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767862/real 1775767865]  req@ffff9f14ab4ff100 x1862022114788864/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767887 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767865.355236:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff100 x1862022114788864/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767887 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767865.355251:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767865.355256:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767865.355260:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114788864:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775767865.355267:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767862/real 1775767865]  req@ffff9f14ab4fce00 x1862022114788608/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767887 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767865.355280:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fce00 x1862022114788608/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767887 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767865.355288:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767865.355292:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767865.355295:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114788608:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775767865.355303:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767862/real 1775767865]  req@ffff9f14ab4fdc00 x1862022114788736/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767887 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767865.355315:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdc00 x1862022114788736/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767887 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767865.355323:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767865.355327:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767865.355330:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114788736:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775767865.355337:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767862/real 1775767865]  req@ffff9f14ab4fd880 x1862022114788224/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767887 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767865.355351:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fd880 x1862022114788224/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767887 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767865.355360:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767865.355364:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767865.355368:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114788224:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775767865.355374:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767862/real 1775767865]  req@ffff9f14ab4fe300 x1862022114788352/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767887 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767865.355387:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fe300 x1862022114788352/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767887 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767865.355413:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767865.355417:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767865.355420:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114788352:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775767865.355464:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767862/real 1775767865]  req@ffff9f14ab4fdf80 x1862022114788480/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767887 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767865.355478:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdf80 x1862022114788480/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767887 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767865.355486:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767865.355490:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767865.355493:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114788480:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775767865.355500:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767862/real 1775767865]  req@ffff9f14ab4fea00 x1862022114788096/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767887 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767865.355514:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fea00 x1862022114788096/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767887 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767865.355522:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767865.355526:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767865.355529:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114788096:192.168.202.110@tcp:250:kworker.0
00010000:00010000:3.0:1775767865.610721:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767866.378686:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775767866.634697:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1775767867.402640:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767867.402648:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775767867.402653:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767867.402662:0:8:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5244
00000100:00080000:3.0:1775767867.402671:0:8:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767867.402677:0:8:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775767867.402711:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:3.0:1775767867.402711:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745500 x1862022114789120/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767867.402735:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767867.402738:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775767867.402739:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767867.402742:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5244
00000100:00080000:3.0:1775767867.402744:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767867.402747:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767867.402756:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767867.402764:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746680 x1862022114789248/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767867.402774:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:1.0:1775767867.402775:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114789120:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775767867.402776:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775767867.402780:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767867.402786:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5244
00000100:00080000:3.0:1775767867.402789:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767867.402792:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767867.402796:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767867.402808:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745c00 x1862022114789376/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767867.402818:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767867.402822:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775767867.402823:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767867.402828:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5244
00000100:00080000:3.0:1775767867.402832:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767867.402836:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767867.402842:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:1.0:1775767867.402849:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767867.402857:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745180 x1862022114789504/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767867.402867:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767867.402870:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775767867.402872:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767867.402877:0:8:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5244
00000100:00080000:3.0:1775767867.402882:0:8:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767867.402886:0:8:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767867.402892:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775767867.402897:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767867/real 1775767867]  req@ffff9f1485745500 x1862022114789120/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767892 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767867.402908:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747100 x1862022114789632/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:1.0:1775767867.402927:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745500 x1862022114789120/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767892 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767867.402928:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767867.402937:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775767867.402941:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767867.402950:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5244
00000100:00080000:1.0:1775767867.402952:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767867.402955:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767867.402960:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:3.0:1775767867.402962:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775767867.402968:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114789120:192.168.202.110@tcp:250:kworker.0
00000008:00100000:3.0:1775767867.402970:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767867.402986:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744000 x1862022114789760/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767867.403003:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:1.0:1775767867.403008:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114789248:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775767867.403009:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775767867.403012:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767867.403021:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5244
00000100:00080000:3.0:1775767867.403025:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767867.403029:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767867.403036:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767867.403050:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747800 x1862022114789888/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767867.403062:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000800:00000100:1.0:1775767867.403063:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775767867.403069:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775767867.403071:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767867.403077:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5244
00000100:00080000:3.0:1775767867.403083:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:1.0:1775767867.403086:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767867/real 1775767867]  req@ffff9f1485746680 x1862022114789248/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767892 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767867.403090:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767867.403099:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775767867.403107:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746680 x1862022114789248/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767892 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767867.403115:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745f80 x1862022114790016/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767867.403120:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767867.403130:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767867.403138:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114789248:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775767867.403146:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114789376:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775767867.403169:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767867.403181:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767867/real 1775767867]  req@ffff9f1485745c00 x1862022114789376/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767892 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767867.403192:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745c00 x1862022114789376/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767892 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767867.403200:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767867.403203:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767867.403207:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114789376:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775767867.403214:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114789504:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775767867.403237:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767867.403248:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767867/real 1775767867]  req@ffff9f1485745180 x1862022114789504/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767892 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767867.403260:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745180 x1862022114789504/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767892 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767867.403268:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767867.403271:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767867.403274:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114789504:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775767867.403287:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114789632:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775767867.403313:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767867.403323:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767867/real 1775767867]  req@ffff9f1485747100 x1862022114789632/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767892 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767867.403335:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747100 x1862022114789632/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767892 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767867.403342:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767867.403345:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767867.403348:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114789632:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775767867.403353:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114789760:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775767867.403379:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767867.403390:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767867/real 1775767867]  req@ffff9f1485744000 x1862022114789760/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767892 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767867.403415:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744000 x1862022114789760/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767892 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767867.403423:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767867.403425:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767867.403428:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114789760:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775767867.403450:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114789888:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775767867.403476:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767867.403488:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767867/real 1775767867]  req@ffff9f1485747800 x1862022114789888/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767892 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767867.403500:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747800 x1862022114789888/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767892 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767867.403515:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767867.403518:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767867.403522:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114789888:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775767867.403528:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114790016:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775767867.403548:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767867.403596:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767867/real 1775767867]  req@ffff9f1485745f80 x1862022114790016/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767892 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767867.403607:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745f80 x1862022114790016/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767892 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767867.403615:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767867.403618:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767867.403622:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114790016:192.168.202.110@tcp:8:kworker.0
00010000:00010000:3.0:1775767867.658748:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767868.426745:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767868.682702:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767869.450625:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767869.706733:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767870.474644:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767870.730679:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767871.498647:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775767871.754706:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775767872.522659:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:2.0:1775767872.522660:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767872.522670:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1775767872.522675:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767872.522684:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5250
00000100:00080000:2.0:1775767872.522696:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767872.522703:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775767872.522742:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3b80 x1862022114790144/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767872.522768:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767872.522771:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775767872.522773:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767872.522776:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5250
00000100:00080000:2.0:1775767872.522779:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767872.522782:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767872.522792:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775767872.522802:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2d80 x1862022114790272/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767872.522812:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767872.522815:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775767872.522816:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767872.522819:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5250
00000100:00080000:2.0:1775767872.522823:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00100000:1.0:1775767872.522825:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114790144:192.168.202.110@tcp:250:kworker.0
00000100:00080000:2.0:1775767872.522829:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767872.522838:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775767872.522846:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1500 x1862022114790400/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767872.522854:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767872.522857:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775767872.522858:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767872.522862:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5250
00000100:00080000:2.0:1775767872.522864:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767872.522867:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767872.522871:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775767872.522879:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2a00 x1862022114790528/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767872.522887:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767872.522889:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1775767872.522890:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000800:00000100:1.0:1775767872.522896:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:2.0:1775767872.522897:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5250
00000100:00080000:2.0:1775767872.522903:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767872.522905:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767872.522909:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775767872.522919:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0380 x1862022114790656/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767872.522926:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767872.522929:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775767872.522931:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767872.522936:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5250
00000100:00080000:2.0:1775767872.522939:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:1.0:1775767872.522942:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767872/real 1775767872]  req@ffff9f14852d3b80 x1862022114790144/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767897 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767872.522946:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767872.522953:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775767872.522967:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0700 x1862022114790784/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:1.0:1775767872.522971:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3b80 x1862022114790144/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767897 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767872.523004:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767872.523005:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767872.523011:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:2.0:1775767872.523015:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00100000:1.0:1775767872.523019:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114790144:192.168.202.110@tcp:250:kworker.0
00000100:00080000:2.0:1775767872.523020:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:1.0:1775767872.523029:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114790272:192.168.202.110@tcp:38:kworker.0
00000100:00080000:2.0:1775767872.523032:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5250
00000100:00080000:2.0:1775767872.523041:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767872.523046:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767872.523054:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000800:00000100:1.0:1775767872.523055:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767872.523068:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767872/real 1775767872]  req@ffff9f14852d2d80 x1862022114790272/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767897 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767872.523075:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0a80 x1862022114790912/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:1.0:1775767872.523093:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2d80 x1862022114790272/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767897 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767872.523093:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767872.523103:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775767872.523107:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767872.523117:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767872.523118:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5250
00000100:00080000:1.0:1775767872.523126:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00080000:2.0:1775767872.523126:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767872.523134:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775767872.523135:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114790272:192.168.202.110@tcp:38:kworker.0
00000008:00100000:2.0:1775767872.523143:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775767872.523155:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3800 x1862022114791040/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:1.0:1775767872.523165:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114790400:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775767872.523183:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767872.523192:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767872/real 1775767872]  req@ffff9f14852d1500 x1862022114790400/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767897 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767872.523200:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1500 x1862022114790400/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767897 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767872.523206:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767872.523209:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767872.523211:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114790400:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775767872.523215:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114790528:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775767872.523226:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767872.523233:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767872/real 1775767872]  req@ffff9f14852d2a00 x1862022114790528/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767897 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767872.523241:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2a00 x1862022114790528/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767897 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767872.523247:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767872.523249:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767872.523251:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114790528:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775767872.523266:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114790656:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775767872.523279:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767872.523286:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767872/real 1775767872]  req@ffff9f14852d0380 x1862022114790656/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767897 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767872.523295:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0380 x1862022114790656/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767897 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767872.523301:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767872.523303:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767872.523305:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114790656:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775767872.523309:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114790784:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775767872.523320:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767872.523326:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767872/real 1775767872]  req@ffff9f14852d0700 x1862022114790784/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767897 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767872.523333:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0700 x1862022114790784/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767897 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767872.523340:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767872.523342:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767872.523345:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114790784:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775767872.523349:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114790912:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775767872.523401:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767872.523414:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767872/real 1775767872]  req@ffff9f14852d0a80 x1862022114790912/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767897 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767872.523428:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0a80 x1862022114790912/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767897 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767872.523448:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767872.523452:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767872.523456:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114790912:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775767872.523475:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114791040:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775767872.523492:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767872.523501:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767872/real 1775767872]  req@ffff9f14852d3800 x1862022114791040/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767897 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767872.523519:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3800 x1862022114791040/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767897 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767872.523525:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767872.523527:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767872.523530:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114791040:192.168.202.110@tcp:8:kworker.0
00010000:00010000:3.0:1775767872.778718:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775767873.546643:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775767873.802699:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775767874.570653:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767874.826697:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767875.594716:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767875.850748:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767876.618671:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767876.874735:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775767877.642683:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:2.0:1775767877.642694:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767877.642703:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1775767877.642707:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767877.642718:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5255
00000100:00080000:2.0:1775767877.642732:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767877.642739:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775767877.642785:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2680 x1862022114791168/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767877.642812:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767877.642816:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775767877.642817:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767877.642821:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5255
00000100:00080000:2.0:1775767877.642826:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767877.642830:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767877.642841:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775767877.642855:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1f80 x1862022114791296/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775767877.642859:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114791168:192.168.202.110@tcp:250:kworker.0
00000100:00080000:2.0:1775767877.642889:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767877.642893:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775767877.642895:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767877.642902:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5255
00000100:00080000:2.0:1775767877.642906:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767877.642910:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767877.642916:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775767877.642931:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3800 x1862022114791424/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767877.642940:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767877.642943:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775767877.642945:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767877.642955:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5255
00000800:00000100:3.0:1775767877.642957:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:2.0:1775767877.642965:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767877.642971:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767877.642978:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775767877.642992:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0a80 x1862022114791552/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767877.643004:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:3.0:1775767877.643009:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767877/real 1775767877]  req@ffff9f14852d2680 x1862022114791168/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767902 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767877.643011:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1775767877.643016:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767877.643033:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5255
00000100:00080000:2.0:1775767877.643039:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767877.643043:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767877.643049:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775767877.643050:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2680 x1862022114791168/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767902 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767877.643072:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0700 x1862022114791680/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767877.643072:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767877.643090:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:2.0:1775767877.643104:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:3.0:1775767877.643105:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114791168:192.168.202.110@tcp:250:kworker.0
00000100:00080000:2.0:1775767877.643111:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775767877.643113:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767877.643117:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5255
00000100:00080000:2.0:1775767877.643119:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767877.643122:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767877.643124:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775767877.643130:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0380 x1862022114791808/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767877.643135:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767877.643137:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775767877.643138:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767877.643141:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5255
00000100:00080000:2.0:1775767877.643143:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767877.643145:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767877.643148:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775767877.643154:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2a00 x1862022114791936/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767877.643160:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767877.643163:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775767877.643164:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767877.643178:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5255
00000100:00080000:2.0:1775767877.643180:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767877.643184:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767877.643187:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775767877.643194:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1500 x1862022114792064/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775767877.643208:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114791296:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775767877.643403:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767877.643460:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767877/real 1775767877]  req@ffff9f14852d1f80 x1862022114791296/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767902 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767877.643472:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1f80 x1862022114791296/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767902 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767877.643479:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767877.643483:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767877.643486:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114791296:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767877.643497:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114791424:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775767877.643527:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767877.643542:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767877/real 1775767877]  req@ffff9f14852d3800 x1862022114791424/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767902 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767877.643550:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3800 x1862022114791424/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767902 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767877.643588:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767877.643593:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767877.643601:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114791424:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767877.643612:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114791552:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775767877.643636:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767877.643649:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767877/real 1775767877]  req@ffff9f14852d0a80 x1862022114791552/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767902 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767877.643670:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0a80 x1862022114791552/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767902 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767877.643678:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767877.643682:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767877.643685:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114791552:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775767877.643692:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114791680:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775767877.643713:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767877.643725:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767877/real 1775767877]  req@ffff9f14852d0700 x1862022114791680/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767902 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767877.643737:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0700 x1862022114791680/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767902 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767877.643743:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767877.643746:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767877.643748:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114791680:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775767877.643753:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114791808:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775767877.643775:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767877.643786:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767877/real 1775767877]  req@ffff9f14852d0380 x1862022114791808/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767902 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767877.643799:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0380 x1862022114791808/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767902 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767877.643814:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767877.643817:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767877.643820:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114791808:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767877.643824:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114791936:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775767877.643840:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767877.643847:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767877/real 1775767877]  req@ffff9f14852d2a00 x1862022114791936/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767902 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767877.643855:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2a00 x1862022114791936/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767902 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767877.643860:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767877.643862:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767877.643864:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114791936:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767877.643879:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114792064:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775767877.643890:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767877.643895:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767877/real 1775767877]  req@ffff9f14852d1500 x1862022114792064/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767902 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767877.643903:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1500 x1862022114792064/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767902 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767877.643908:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767877.643910:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767877.643912:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114792064:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775767877.898727:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775767878.666660:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775767878.922680:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775767879.690644:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767879.946700:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775767880.714655:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775767880.970768:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775767881.738631:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767881.994709:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775767882.762659:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:2.0:1775767882.762662:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767882.762671:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1775767882.762686:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767882.762697:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5260
00000100:00080000:2.0:1775767882.762708:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767882.762715:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775767882.762758:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2d80 x1862022114792192/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767882.762788:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767882.762793:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775767882.762795:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767882.762800:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5260
00000100:00080000:2.0:1775767882.762805:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767882.762811:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775767882.762819:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114792192:192.168.202.110@tcp:250:kworker.0
00000008:00100000:2.0:1775767882.762822:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775767882.762840:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3b80 x1862022114792320/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767882.762852:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767882.762857:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775767882.762858:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767882.762862:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5260
00000100:00080000:2.0:1775767882.762867:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767882.762871:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767882.762876:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000800:00000100:3.0:1775767882.762882:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767882.762891:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0000 x1862022114792448/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767882.762904:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767882.762907:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775767882.762909:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767882.762914:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5260
00000100:00080000:2.0:1775767882.762919:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767882.762923:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775767882.762928:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767882/real 1775767882]  req@ffff9f14852d2d80 x1862022114792192/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767907 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:2.0:1775767882.762930:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775767882.762946:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1180 x1862022114792576/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775767882.762951:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2d80 x1862022114792192/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767907 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767882.762971:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767882.762974:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767882.762978:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775767882.762981:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:2.0:1775767882.762982:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:3.0:1775767882.762986:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114792192:192.168.202.110@tcp:250:kworker.0
00000100:00080000:2.0:1775767882.762992:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5260
00000100:00080000:2.0:1775767882.762998:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767882.763002:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767882.763007:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775767882.763011:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114792320:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767882.763024:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1880 x1862022114792704/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:3.0:1775767882.763035:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:2.0:1775767882.763038:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767882.763043:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775767882.763045:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:3.0:1775767882.763047:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767882/real 1775767882]  req@ffff9f14852d3b80 x1862022114792320/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767907 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767882.763053:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5260
00000100:00080000:2.0:1775767882.763060:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:3.0:1775767882.763065:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3b80 x1862022114792320/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767907 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767882.763068:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767882.763077:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00080000:3.0:1775767882.763079:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767882.763084:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767882.763087:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114792320:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767882.763092:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0e00 x1862022114792832/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775767882.763094:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114792448:192.168.202.110@tcp:38:kworker.0
00000100:00080000:2.0:1775767882.763116:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767882.763119:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775767882.763122:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000800:00000100:3.0:1775767882.763124:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:2.0:1775767882.763131:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5260
00000100:00080000:2.0:1775767882.763136:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:3.0:1775767882.763137:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767882/real 1775767882]  req@ffff9f14852d0000 x1862022114792448/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767907 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767882.763144:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767882.763153:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767882.763154:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0000 x1862022114792448/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767907 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767882.763164:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767882.763167:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767882.763171:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2300 x1862022114792960/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775767882.763172:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114792448:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767882.763182:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114792576:192.168.202.110@tcp:8:kworker.0
00000100:00080000:2.0:1775767882.763193:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767882.763197:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775767882.763199:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000800:00000100:3.0:1775767882.763202:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:2.0:1775767882.763208:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5260
00000100:00080000:2.0:1775767882.763213:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767882.763218:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775767882.763221:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767882/real 1775767882]  req@ffff9f14852d1180 x1862022114792576/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767907 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:2.0:1775767882.763227:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775767882.763232:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1180 x1862022114792576/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767907 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767882.763238:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767882.763240:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767882.763242:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114792576:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775767882.763267:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3100 x1862022114793088/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775767882.763279:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114792704:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775767882.763298:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767882.763307:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767882/real 1775767882]  req@ffff9f14852d1880 x1862022114792704/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767907 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767882.763314:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1880 x1862022114792704/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767907 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767882.763319:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767882.763322:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767882.763324:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114792704:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775767882.763328:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114792832:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775767882.763337:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767882.763343:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767882/real 1775767882]  req@ffff9f14852d0e00 x1862022114792832/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767907 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767882.763350:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0e00 x1862022114792832/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767907 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767882.763355:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767882.763357:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767882.763358:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114792832:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767882.763362:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114792960:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775767882.763371:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767882.763380:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767882/real 1775767882]  req@ffff9f14852d2300 x1862022114792960/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767907 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767882.763398:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2300 x1862022114792960/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767907 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767882.763407:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767882.763410:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767882.763414:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114792960:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767882.763429:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114793088:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775767882.763447:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767882.763455:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767882/real 1775767882]  req@ffff9f14852d3100 x1862022114793088/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767907 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767882.763463:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3100 x1862022114793088/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767907 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767882.763468:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767882.763470:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767882.763473:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114793088:192.168.202.110@tcp:8:kworker.0
00010000:00010000:3.0:1775767883.018773:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775767883.786627:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767884.042702:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775767884.810699:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767885.066733:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767885.834650:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775767886.090698:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767886.858635:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767887.114750:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767887.882698:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:1.0:1775767887.882733:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767887.882744:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1775767887.882749:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767887.882760:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5265
00000100:00080000:1.0:1775767887.882773:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767887.882781:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775767887.882822:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fea00 x1862022114793216/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767887.882850:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767887.882853:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775767887.882854:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767887.882857:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5265
00000100:00080000:1.0:1775767887.882860:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767887.882863:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767887.882872:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767887.882888:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdf80 x1862022114793344/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767887.882896:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767887.882899:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775767887.882900:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767887.882905:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5265
00000100:00080000:1.0:1775767887.882908:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00100000:3.0:1775767887.882908:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114793216:192.168.202.110@tcp:250:kworker.0
00000100:00080000:1.0:1775767887.882913:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767887.882919:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767887.882926:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fe300 x1862022114793472/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767887.882932:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767887.882935:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775767887.882936:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767887.882938:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5265
00000100:00080000:1.0:1775767887.882940:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767887.882942:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767887.882945:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775767887.882952:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fd880 x1862022114793600/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767887.882958:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767887.882960:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1775767887.882961:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767887.882963:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5265
00000100:00080000:1.0:1775767887.882966:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767887.882968:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767887.882972:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:3.0:1775767887.882977:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767887.882986:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdc00 x1862022114793728/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767887.882994:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767887.882996:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775767887.882998:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767887.883001:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5265
00000100:00080000:1.0:1775767887.883003:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767887.883006:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767887.883011:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767887.883015:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767887/real 1775767887]  req@ffff9f14ab4fea00 x1862022114793216/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767912 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767887.883026:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fce00 x1862022114793856/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775767887.883033:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fea00 x1862022114793216/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767912 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767887.883045:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767887.883048:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767887.883049:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775767887.883053:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767887.883054:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767887.883060:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114793216:192.168.202.110@tcp:250:kworker.0
00000100:00080000:1.0:1775767887.883063:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5265
00000100:00080000:1.0:1775767887.883070:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767887.883074:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767887.883079:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767887.883089:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114793344:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775767887.883092:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff100 x1862022114793984/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767887.883103:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767887.883106:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775767887.883107:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767887.883110:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5265
00000100:00080000:1.0:1775767887.883114:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767887.883117:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767887.883122:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:3.0:1775767887.883124:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767887.883136:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fc380 x1862022114794112/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775767887.883141:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767887/real 1775767887]  req@ffff9f14ab4fdf80 x1862022114793344/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767912 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767887.883161:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdf80 x1862022114793344/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767912 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767887.883167:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767887.883169:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767887.883171:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114793344:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767887.883176:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114793472:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775767887.883194:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767887.883217:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767887/real 1775767887]  req@ffff9f14ab4fe300 x1862022114793472/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767912 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767887.883232:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fe300 x1862022114793472/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767912 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767887.883248:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767887.883252:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767887.883255:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114793472:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767887.883261:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114793600:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775767887.883278:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767887.883285:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767887/real 1775767887]  req@ffff9f14ab4fd880 x1862022114793600/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767912 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767887.883293:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fd880 x1862022114793600/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767912 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767887.883298:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767887.883300:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767887.883302:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114793600:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775767887.883306:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114793728:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775767887.883316:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767887.883321:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767887/real 1775767887]  req@ffff9f14ab4fdc00 x1862022114793728/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767912 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767887.883328:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdc00 x1862022114793728/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767912 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767887.883333:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767887.883334:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767887.883337:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114793728:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775767887.883341:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114793856:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775767887.883350:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767887.883355:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767887/real 1775767887]  req@ffff9f14ab4fce00 x1862022114793856/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767912 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767887.883362:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fce00 x1862022114793856/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767912 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767887.883367:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767887.883369:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767887.883370:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114793856:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767887.883383:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114793984:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775767887.883392:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767887.883398:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767887/real 1775767887]  req@ffff9f14ab4ff100 x1862022114793984/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767912 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767887.883406:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff100 x1862022114793984/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767912 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767887.883411:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767887.883412:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767887.883414:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114793984:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767887.883417:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114794112:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775767887.883426:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767887.883432:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767887/real 1775767887]  req@ffff9f14ab4fc380 x1862022114794112/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767912 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767887.883439:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fc380 x1862022114794112/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767912 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767887.883444:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767887.883446:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767887.883447:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114794112:192.168.202.110@tcp:8:kworker.0
00010000:00010000:1.0:1775767888.138749:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767888.906645:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767889.162788:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767889.930630:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767890.186698:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767890.954647:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767891.210748:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775767891.978679:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767892.234737:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1775767893.002651:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:3.0:1775767893.002654:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:1.0:1775767893.002660:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1775767893.002666:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767893.002674:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5270
00000100:00080000:1.0:1775767893.002684:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767893.002690:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775767893.002727:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fca80 x1862022114794240/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767893.002750:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767893.002753:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775767893.002754:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767893.002757:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5270
00000100:00080000:1.0:1775767893.002760:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767893.002763:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767893.002772:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767893.002780:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114794240:192.168.202.110@tcp:250:kworker.0
00000100:00100000:1.0:1775767893.002783:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ffb80 x1862022114794368/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767893.002797:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767893.002800:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775767893.002801:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767893.002804:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5270
00000100:00080000:1.0:1775767893.002807:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767893.002810:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767893.002813:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767893.002820:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fed80 x1862022114794496/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767893.002826:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767893.002829:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775767893.002830:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767893.002832:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5270
00000100:00080000:1.0:1775767893.002835:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767893.002839:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000800:00000100:3.0:1775767893.002839:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000008:00100000:1.0:1775767893.002846:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775767893.002853:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff800 x1862022114794624/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767893.002860:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767893.002862:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1775767893.002863:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767893.002865:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5270
00000100:00080000:1.0:1775767893.002869:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767893.002873:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767893.002878:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775767893.002878:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767893/real 1775767893]  req@ffff9f14ab4fca80 x1862022114794240/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767918 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767893.002899:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fc380 x1862022114794752/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775767893.002900:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fca80 x1862022114794240/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767918 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767893.002932:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767893.002934:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767893.002937:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767893.002942:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114794240:192.168.202.110@tcp:250:kworker.0
00000100:00080000:1.0:1775767893.002943:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775767893.002947:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767893.002952:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5270
00000100:00080000:1.0:1775767893.002957:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767893.002960:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775767893.002964:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ffb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114794368:192.168.202.110@tcp:38:kworker.0
00000008:00100000:1.0:1775767893.002965:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767893.002982:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff100 x1862022114794880/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:3.0:1775767893.002988:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:1.0:1775767893.002998:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767893.003000:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775767893.003002:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:3.0:1775767893.003004:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767893/real 1775767893]  req@ffff9f14ab4ffb80 x1862022114794368/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767918 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767893.003008:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5270
00000100:00080000:1.0:1775767893.003014:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767893.003020:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767893.003023:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767893.003024:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ffb80 x1862022114794368/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767918 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767893.003032:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00100000:1.0:1775767893.003034:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fce00 x1862022114795008/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767893.003037:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767893.003044:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ffb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114794368:192.168.202.110@tcp:38:kworker.0
00000100:00080000:1.0:1775767893.003054:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767893.003056:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775767893.003059:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:3.0:1775767893.003064:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114794496:192.168.202.110@tcp:38:kworker.0
00000100:00080000:1.0:1775767893.003065:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5270
00000100:00080000:1.0:1775767893.003070:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767893.003073:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767893.003076:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775767893.003086:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdc00 x1862022114795136/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:3.0:1775767893.003096:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767893.003108:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767893/real 1775767893]  req@ffff9f14ab4fed80 x1862022114794496/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767918 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767893.003122:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fed80 x1862022114794496/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767918 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767893.003131:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767893.003134:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767893.003138:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114794496:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767893.003145:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114794624:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775767893.003181:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767893.003191:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767893/real 1775767893]  req@ffff9f14ab4ff800 x1862022114794624/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767918 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767893.003200:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff800 x1862022114794624/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767918 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767893.003206:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767893.003208:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767893.003210:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114794624:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775767893.003214:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114794752:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775767893.003227:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767893.003232:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767893/real 1775767893]  req@ffff9f14ab4fc380 x1862022114794752/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767918 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767893.003239:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fc380 x1862022114794752/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767918 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767893.003244:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767893.003245:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767893.003247:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114794752:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775767893.003259:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114794880:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775767893.003269:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767893.003275:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767893/real 1775767893]  req@ffff9f14ab4ff100 x1862022114794880/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767918 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767893.003282:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff100 x1862022114794880/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767918 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767893.003287:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767893.003289:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767893.003291:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114794880:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767893.003295:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114795008:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775767893.003304:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767893.003309:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767893/real 1775767893]  req@ffff9f14ab4fce00 x1862022114795008/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767918 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767893.003315:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fce00 x1862022114795008/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767918 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767893.003320:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767893.003322:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767893.003324:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114795008:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767893.003327:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114795136:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775767893.003335:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767893.003340:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767893/real 1775767893]  req@ffff9f14ab4fdc00 x1862022114795136/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767918 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767893.003348:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdc00 x1862022114795136/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767918 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767893.003352:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767893.003354:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767893.003356:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114795136:192.168.202.110@tcp:8:kworker.0
00010000:00010000:1.0:1775767893.258760:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775767894.026651:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767894.282788:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775767895.050678:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775767895.306800:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767896.074697:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767896.330713:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767897.098671:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767897.354729:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767898.122651:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:1.0:1775767898.122652:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767898.122661:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1775767898.122666:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767898.122677:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5275
00000100:00080000:1.0:1775767898.122689:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767898.122696:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775767898.122750:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fd880 x1862022114795264/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767898.122782:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767898.122786:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775767898.122788:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767898.122792:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5275
00000100:00080000:1.0:1775767898.122797:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767898.122802:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767898.122813:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767898.122829:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fe300 x1862022114795392/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775767898.122834:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114795264:192.168.202.110@tcp:250:kworker.0
00000100:00080000:1.0:1775767898.122851:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767898.122854:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775767898.122856:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767898.122861:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5275
00000100:00080000:1.0:1775767898.122866:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767898.122870:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767898.122876:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767898.122890:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdf80 x1862022114795520/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767898.122902:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000800:00000100:3.0:1775767898.122904:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:1.0:1775767898.122908:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775767898.122911:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767898.122916:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5275
00000100:00080000:1.0:1775767898.122922:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767898.122926:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767898.122931:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775767898.122944:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fea00 x1862022114795648/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775767898.122951:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767898/real 1775767898]  req@ffff9f14ab4fd880 x1862022114795264/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767923 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767898.122971:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767898.122974:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1775767898.122975:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767898.122978:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5275
00000100:00080000:1.0:1775767898.122981:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767898.122984:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767898.122988:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775767898.122997:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fd880 x1862022114795264/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767923 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767898.122999:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdc00 x1862022114795776/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767898.123032:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767898.123033:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767898.123040:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775767898.123042:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:1.0:1775767898.123044:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:3.0:1775767898.123049:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114795264:192.168.202.110@tcp:250:kworker.0
00000100:00080000:1.0:1775767898.123053:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5275
00000100:00080000:1.0:1775767898.123060:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767898.123065:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767898.123071:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767898.123078:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114795392:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775767898.123084:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fce00 x1862022114795904/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767898.123097:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767898.123101:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775767898.123103:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767898.123125:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5275
00000100:00080000:1.0:1775767898.123130:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000800:00000100:3.0:1775767898.123135:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:1.0:1775767898.123139:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767898.123146:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767898.123154:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767898/real 1775767898]  req@ffff9f14ab4fe300 x1862022114795392/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767923 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767898.123162:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff100 x1862022114796032/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775767898.123171:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fe300 x1862022114795392/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767923 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767898.123180:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767898.123182:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767898.123188:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775767898.123189:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00080000:1.0:1775767898.123193:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:3.0:1775767898.123195:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114795392:192.168.202.110@tcp:38:kworker.0
00000100:00080000:1.0:1775767898.123203:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5275
00000100:00100000:3.0:1775767898.123205:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114795520:192.168.202.110@tcp:38:kworker.0
00000100:00080000:1.0:1775767898.123209:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767898.123217:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767898.123222:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:3.0:1775767898.123231:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767898.123242:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fc380 x1862022114796160/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775767898.123243:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767898/real 1775767898]  req@ffff9f14ab4fdf80 x1862022114795520/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767923 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767898.123278:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdf80 x1862022114795520/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767923 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767898.123285:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767898.123288:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767898.123291:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114795520:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767898.123296:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114795648:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775767898.123309:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767898.123315:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767898/real 1775767898]  req@ffff9f14ab4fea00 x1862022114795648/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767923 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767898.123322:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fea00 x1862022114795648/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767923 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767898.123327:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767898.123329:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767898.123331:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114795648:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775767898.123334:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114795776:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775767898.123344:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767898.123349:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767898/real 1775767898]  req@ffff9f14ab4fdc00 x1862022114795776/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767923 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767898.123361:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdc00 x1862022114795776/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767923 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767898.123365:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767898.123367:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767898.123369:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114795776:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775767898.123380:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114795904:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775767898.123389:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767898.123395:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767898/real 1775767898]  req@ffff9f14ab4fce00 x1862022114795904/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767923 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767898.123401:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fce00 x1862022114795904/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767923 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767898.123406:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767898.123408:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767898.123409:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114795904:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767898.123413:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114796032:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775767898.123421:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767898.123427:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767898/real 1775767898]  req@ffff9f14ab4ff100 x1862022114796032/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767923 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767898.123434:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff100 x1862022114796032/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767923 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767898.123443:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767898.123444:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767898.123446:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114796032:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767898.123450:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114796160:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775767898.123458:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767898.123463:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767898/real 1775767898]  req@ffff9f14ab4fc380 x1862022114796160/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767923 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767898.123469:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fc380 x1862022114796160/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767923 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767898.123474:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767898.123476:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767898.123478:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114796160:192.168.202.110@tcp:8:kworker.0
00010000:00010000:3.0:1775767898.378744:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767899.146675:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767899.402743:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767900.170669:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775767900.426688:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767901.194652:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767901.450723:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775767902.218699:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775767902.474723:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775767903.242624:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:1.0:1775767903.242645:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767903.242650:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1775767903.242655:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767903.242663:0:8:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5280
00000100:00080000:1.0:1775767903.242674:0:8:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767903.242679:0:8:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775767903.242716:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff800 x1862022114796288/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767903.242743:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767903.242745:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775767903.242747:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767903.242750:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5280
00000100:00080000:1.0:1775767903.242752:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767903.242754:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767903.242762:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767903.242773:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fed80 x1862022114796416/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767903.242782:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767903.242786:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775767903.242787:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:3.0:1775767903.242787:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114796288:192.168.202.110@tcp:250:kworker.0
00000100:00080000:1.0:1775767903.242795:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5280
00000100:00080000:1.0:1775767903.242798:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767903.242801:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767903.242804:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767903.242811:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ffb80 x1862022114796544/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767903.242817:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767903.242819:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775767903.242820:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767903.242823:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5280
00000100:00080000:1.0:1775767903.242826:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767903.242829:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767903.242832:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775767903.242839:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fca80 x1862022114796672/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767903.242844:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767903.242847:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1775767903.242848:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767903.242851:0:8:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5280
00000100:00080000:1.0:1775767903.242853:0:8:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767903.242855:0:8:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767903.242858:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:3.0:1775767903.242861:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767903.242871:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fc380 x1862022114796800/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767903.242881:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767903.242885:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775767903.242887:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767903.242892:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5280
00000100:00080000:1.0:1775767903.242896:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:3.0:1775767903.242900:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767903/real 1775767903]  req@ffff9f14ab4ff800 x1862022114796288/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767928 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767903.242906:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767903.242916:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767903.242924:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff800 x1862022114796288/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767928 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767903.242937:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00100000:1.0:1775767903.242942:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff100 x1862022114796928/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767903.242943:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767903.242950:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114796288:192.168.202.110@tcp:250:kworker.0
00000100:00080000:1.0:1775767903.242970:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767903.242972:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775767903.242973:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767903.242978:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5280
00000100:00100000:3.0:1775767903.242980:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114796416:192.168.202.110@tcp:38:kworker.0
00000100:00080000:1.0:1775767903.242982:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767903.242989:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767903.242995:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767903.243006:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fce00 x1862022114797056/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767903.243014:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767903.243020:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000800:00000100:3.0:1775767903.243020:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:1.0:1775767903.243024:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767903.243028:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5280
00000100:00080000:1.0:1775767903.243033:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767903.243036:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767903.243039:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775767903.243041:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767903/real 1775767903]  req@ffff9f14ab4fed80 x1862022114796416/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767928 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767903.243050:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdc00 x1862022114797184/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775767903.243076:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fed80 x1862022114796416/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767928 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767903.243086:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767903.243089:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767903.243092:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114796416:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767903.243098:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ffb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114796544:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775767903.243114:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767903.243121:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767903/real 1775767903]  req@ffff9f14ab4ffb80 x1862022114796544/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767928 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767903.243129:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ffb80 x1862022114796544/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767928 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767903.243134:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767903.243136:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767903.243138:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ffb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114796544:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767903.243143:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114796672:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775767903.243153:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767903.243158:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767903/real 1775767903]  req@ffff9f14ab4fca80 x1862022114796672/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767928 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767903.243169:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fca80 x1862022114796672/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767928 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767903.243177:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767903.243181:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767903.243184:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114796672:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775767903.243190:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114796800:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775767903.243210:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767903.243221:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767903/real 1775767903]  req@ffff9f14ab4fc380 x1862022114796800/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767928 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767903.243239:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fc380 x1862022114796800/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767928 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767903.243244:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767903.243246:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767903.243248:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114796800:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775767903.243253:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114796928:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775767903.243267:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767903.243274:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767903/real 1775767903]  req@ffff9f14ab4ff100 x1862022114796928/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767928 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767903.243282:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff100 x1862022114796928/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767928 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767903.243287:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767903.243289:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767903.243291:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114796928:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767903.243303:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114797056:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775767903.243313:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767903.243319:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767903/real 1775767903]  req@ffff9f14ab4fce00 x1862022114797056/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767928 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767903.243327:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fce00 x1862022114797056/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767928 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767903.243331:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767903.243333:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767903.243335:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114797056:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767903.243339:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114797184:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775767903.243347:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767903.243352:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767903/real 1775767903]  req@ffff9f14ab4fdc00 x1862022114797184/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767928 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767903.243359:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdc00 x1862022114797184/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767928 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767903.243368:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767903.243370:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767903.243372:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114797184:192.168.202.110@tcp:8:kworker.0
00010000:00010000:3.0:1775767903.498754:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775767904.266672:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767904.522716:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775767905.290667:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767905.546731:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767906.314665:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767906.570734:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767907.338702:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767907.594755:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1775767908.362687:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1775767908.362693:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:1.0:1775767908.362699:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1775767908.362703:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767908.362714:0:8:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5285
00000100:00080000:1.0:1775767908.362727:0:8:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767908.362735:0:8:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775767908.362778:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fea00 x1862022114797312/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767908.362805:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767908.362807:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775767908.362809:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767908.362812:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5285
00000100:00080000:1.0:1775767908.362815:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767908.362817:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767908.362827:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767908.362838:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdf80 x1862022114797440/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767908.362846:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767908.362849:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775767908.362851:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:3.0:1775767908.362856:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114797312:192.168.202.110@tcp:250:kworker.0
00000100:00080000:1.0:1775767908.362857:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5285
00000100:00080000:1.0:1775767908.362863:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767908.362866:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767908.362870:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767908.362879:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fe300 x1862022114797568/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767908.362888:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767908.362893:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775767908.362895:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767908.362899:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5285
00000100:00080000:1.0:1775767908.362904:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767908.362909:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767908.362915:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775767908.362928:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fd880 x1862022114797696/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767908.362939:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000800:00000100:3.0:1775767908.362942:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:1.0:1775767908.362944:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1775767908.362947:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767908.362951:0:8:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5285
00000100:00080000:1.0:1775767908.362957:0:8:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767908.362961:0:8:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767908.362965:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775767908.362975:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdc00 x1862022114797824/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767908.362982:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767908.362985:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775767908.362986:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767908.362988:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5285
00000100:00080000:1.0:1775767908.362991:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767908.362995:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767908.363001:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767908.363020:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767908/real 1775767908]  req@ffff9f14ab4fea00 x1862022114797312/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767933 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767908.363031:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fce00 x1862022114797952/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775767908.363043:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fea00 x1862022114797312/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767933 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767908.363044:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767908.363055:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775767908.363060:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767908.363063:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767908.363068:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5285
00000100:00080000:3.0:1775767908.363072:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:1.0:1775767908.363074:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:3.0:1775767908.363077:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114797312:192.168.202.110@tcp:250:kworker.0
00000100:00080000:1.0:1775767908.363082:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767908.363088:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767908.363103:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114797440:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775767908.363109:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff100 x1862022114798080/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767908.363119:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767908.363122:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775767908.363124:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000800:00000100:3.0:1775767908.363128:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:1.0:1775767908.363129:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5285
00000100:00080000:1.0:1775767908.363134:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767908.363138:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767908.363142:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775767908.363146:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767908/real 1775767908]  req@ffff9f14ab4fdf80 x1862022114797440/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767933 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767908.363153:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fc380 x1862022114798208/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775767908.363170:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdf80 x1862022114797440/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767933 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767908.363179:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767908.363182:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767908.363185:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114797440:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767908.363189:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114797568:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775767908.363203:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767908.363210:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767908/real 1775767908]  req@ffff9f14ab4fe300 x1862022114797568/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767933 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767908.363217:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fe300 x1862022114797568/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767933 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767908.363223:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767908.363225:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767908.363227:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114797568:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767908.363230:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114797696:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775767908.363241:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767908.363246:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767908/real 1775767908]  req@ffff9f14ab4fd880 x1862022114797696/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767933 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767908.363253:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fd880 x1862022114797696/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767933 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767908.363258:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767908.363260:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767908.363262:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114797696:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775767908.363266:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114797824:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775767908.363276:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767908.363281:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767908/real 1775767908]  req@ffff9f14ab4fdc00 x1862022114797824/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767933 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767908.363288:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdc00 x1862022114797824/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767933 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767908.363294:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767908.363296:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767908.363298:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114797824:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775767908.363301:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114797952:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775767908.363311:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767908.363317:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767908/real 1775767908]  req@ffff9f14ab4fce00 x1862022114797952/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767933 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767908.363325:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fce00 x1862022114797952/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767933 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767908.363330:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767908.363333:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767908.363336:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114797952:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767908.363355:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114798080:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775767908.363376:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767908.363388:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767908/real 1775767908]  req@ffff9f14ab4ff100 x1862022114798080/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767933 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767908.363400:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff100 x1862022114798080/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767933 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767908.363406:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767908.363408:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767908.363410:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114798080:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767908.363414:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114798208:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775767908.363429:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767908.363444:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767908/real 1775767908]  req@ffff9f14ab4fc380 x1862022114798208/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767933 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767908.363452:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fc380 x1862022114798208/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767933 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767908.363458:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767908.363460:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767908.363462:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114798208:192.168.202.110@tcp:8:kworker.0
00010000:00010000:3.0:1775767908.618746:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767909.386642:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767909.642717:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767910.410665:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767910.666722:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767911.434640:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767911.690718:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767912.458667:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767912.714773:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1775767913.482630:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767913.482639:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1775767913.482645:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00100000:3.0:1775767913.482654:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:1.0:1775767913.482657:0:8:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5290
00000100:00080000:1.0:1775767913.482671:0:8:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767913.482680:0:8:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775767913.482720:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fca80 x1862022114798336/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767913.482744:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767913.482747:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775767913.482749:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767913.482753:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5290
00000100:00080000:1.0:1775767913.482758:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767913.482762:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767913.482772:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767913.482774:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114798336:192.168.202.110@tcp:250:kworker.0
00000100:00100000:1.0:1775767913.482786:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ffb80 x1862022114798464/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767913.482796:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767913.482799:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775767913.482801:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767913.482806:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5290
00000100:00080000:1.0:1775767913.482810:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767913.482813:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767913.482817:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767913.482826:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fed80 x1862022114798592/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:3.0:1775767913.482838:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:1.0:1775767913.482841:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767913.482844:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775767913.482846:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767913.482849:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5290
00000100:00080000:1.0:1775767913.482854:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767913.482856:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767913.482859:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775767913.482869:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff800 x1862022114798720/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775767913.482873:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767913/real 1775767913]  req@ffff9f14ab4fca80 x1862022114798336/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767938 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767913.482883:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767913.482891:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1775767913.482892:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767913.482895:0:8:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5290
00000100:00080000:1.0:1775767913.482897:0:8:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767913.482899:0:8:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767913.482902:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775767913.482907:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fc380 x1862022114798848/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767913.482913:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767913.482916:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775767913.482917:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767913.482919:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5290
00000100:00080000:1.0:1775767913.482921:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767913.482924:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767913.482927:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767913.482932:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff100 x1862022114798976/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767913.482937:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767913.482940:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775767913.482940:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767913.482943:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5290
00000100:00080000:1.0:1775767913.482946:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767913.482949:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767913.482952:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767913.482975:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fce00 x1862022114799104/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775767913.482978:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fca80 x1862022114798336/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767938 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767913.482994:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767913.482997:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767913.482999:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775767913.483003:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767913.483004:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:1.0:1775767913.483011:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5290
00000100:00100000:3.0:1775767913.483012:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114798336:192.168.202.110@tcp:250:kworker.0
00000100:00080000:1.0:1775767913.483020:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767913.483026:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767913.483031:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775767913.483043:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdc00 x1862022114799232/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775767913.483125:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ffb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114798464:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775767913.483198:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767913.483210:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767913/real 1775767913]  req@ffff9f14ab4ffb80 x1862022114798464/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767938 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767913.483219:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ffb80 x1862022114798464/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767938 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767913.483225:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767913.483227:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767913.483229:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ffb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114798464:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767913.483234:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114798592:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775767913.483248:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767913.483253:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767913/real 1775767913]  req@ffff9f14ab4fed80 x1862022114798592/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767938 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767913.483265:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fed80 x1862022114798592/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767938 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767913.483270:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767913.483272:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767913.483274:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114798592:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767913.483278:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114798720:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775767913.483288:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767913.483293:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767913/real 1775767913]  req@ffff9f14ab4ff800 x1862022114798720/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767938 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767913.483301:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff800 x1862022114798720/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767938 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767913.483305:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767913.483307:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767913.483309:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114798720:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775767913.483312:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114798848:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775767913.483321:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767913.483326:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767913/real 1775767913]  req@ffff9f14ab4fc380 x1862022114798848/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767938 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767913.483333:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fc380 x1862022114798848/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767938 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767913.483354:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767913.483356:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767913.483358:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114798848:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775767913.483362:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114798976:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775767913.483373:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767913.483380:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767913/real 1775767913]  req@ffff9f14ab4ff100 x1862022114798976/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767938 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767913.483387:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff100 x1862022114798976/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767938 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767913.483393:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767913.483395:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767913.483396:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114798976:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767913.483400:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114799104:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775767913.483409:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767913.483414:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767913/real 1775767913]  req@ffff9f14ab4fce00 x1862022114799104/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767938 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767913.483421:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fce00 x1862022114799104/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767938 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767913.483426:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767913.483428:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767913.483430:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114799104:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767913.483434:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114799232:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775767913.483443:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767913.483448:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767913/real 1775767913]  req@ffff9f14ab4fdc00 x1862022114799232/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767938 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767913.483455:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdc00 x1862022114799232/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767938 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767913.483460:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767913.483462:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767913.483464:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114799232:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775767913.738732:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775767914.506682:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767914.762775:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775767915.530637:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767915.786772:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767916.554726:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767916.810765:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767917.578669:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767917.834719:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767918.602695:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:2.0:1775767918.602695:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767918.602706:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1775767918.602711:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767918.602721:0:8:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5296
00000100:00080000:2.0:1775767918.602733:0:8:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767918.602740:0:8:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775767918.602791:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3480 x1862022114799360/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767918.602825:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767918.602829:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775767918.602831:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767918.602834:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5296
00000100:00080000:2.0:1775767918.602838:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767918.602841:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767918.602850:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775767918.602869:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1c00 x1862022114799488/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767918.602878:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767918.602882:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775767918.602883:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:3.0:1775767918.602887:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114799360:192.168.202.110@tcp:250:kworker.0
00000100:00080000:2.0:1775767918.602889:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5296
00000100:00080000:2.0:1775767918.602893:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767918.602900:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767918.602932:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775767918.602951:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3100 x1862022114799616/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767918.602963:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767918.602967:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775767918.602968:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767918.602972:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5296
00000100:00080000:2.0:1775767918.602976:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767918.602980:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767918.602985:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:3.0:1775767918.602990:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767918.603005:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2300 x1862022114799744/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767918.603014:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767918.603019:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1775767918.603020:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767918.603025:0:8:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5296
00000100:00080000:2.0:1775767918.603029:0:8:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767918.603032:0:8:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767918.603039:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775767918.603055:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0e00 x1862022114799872/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775767918.603055:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767918/real 1775767918]  req@ffff9f14852d3480 x1862022114799360/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767943 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767918.603089:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767918.603093:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775767918.603096:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:3.0:1775767918.603101:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3480 x1862022114799360/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767943 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767918.603102:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5296
00000100:00080000:2.0:1775767918.603110:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767918.603118:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00080000:3.0:1775767918.603123:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000008:00100000:2.0:1775767918.603125:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00080000:3.0:1775767918.603131:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775767918.603140:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1880 x1862022114800000/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775767918.603140:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114799360:192.168.202.110@tcp:250:kworker.0
00000100:00080000:2.0:1775767918.603156:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767918.603160:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775767918.603162:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767918.603167:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5296
00000100:00080000:2.0:1775767918.603172:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:3.0:1775767918.603176:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114799488:192.168.202.110@tcp:38:kworker.0
00000100:00080000:2.0:1775767918.603180:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767918.603187:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775767918.603201:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1180 x1862022114800128/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:3.0:1775767918.603212:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:2.0:1775767918.603213:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767918.603218:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775767918.603220:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767918.603226:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5296
00000100:00100000:3.0:1775767918.603229:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767918/real 1775767918]  req@ffff9f14852d1c00 x1862022114799488/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767943 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767918.603232:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767918.603241:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767918.603250:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775767918.603254:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1c00 x1862022114799488/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767943 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767918.603264:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00100000:2.0:1775767918.603268:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0000 x1862022114800256/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767918.603272:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767918.603280:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114799488:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767918.603293:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114799616:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775767918.603315:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767918.603325:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767918/real 1775767918]  req@ffff9f14852d3100 x1862022114799616/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767943 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767918.603336:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3100 x1862022114799616/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767943 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767918.603344:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767918.603347:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767918.603350:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114799616:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767918.603356:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114799744:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775767918.603374:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767918.603383:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767918/real 1775767918]  req@ffff9f14852d2300 x1862022114799744/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767943 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767918.603396:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2300 x1862022114799744/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767943 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767918.603405:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767918.603408:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767918.603411:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114799744:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775767918.603417:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114799872:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775767918.603436:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767918.603447:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767918/real 1775767918]  req@ffff9f14852d0e00 x1862022114799872/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767943 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767918.603459:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0e00 x1862022114799872/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767943 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767918.603467:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767918.603471:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767918.603474:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114799872:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775767918.603481:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114800000:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775767918.603500:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767918.603511:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767918/real 1775767918]  req@ffff9f14852d1880 x1862022114800000/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767943 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767918.603523:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1880 x1862022114800000/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767943 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767918.603533:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767918.603535:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767918.603537:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114800000:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767918.603577:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114800128:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775767918.603594:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767918.603602:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767918/real 1775767918]  req@ffff9f14852d1180 x1862022114800128/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767943 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767918.603610:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1180 x1862022114800128/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767943 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767918.603615:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767918.603618:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767918.603620:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114800128:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767918.603623:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114800256:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775767918.603633:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767918.603639:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767918/real 1775767918]  req@ffff9f14852d0000 x1862022114800256/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767943 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767918.603646:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0000 x1862022114800256/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767943 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767918.603651:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767918.603653:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767918.603655:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114800256:192.168.202.110@tcp:8:kworker.0
00010000:00010000:3.0:1775767918.858734:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767919.626719:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767919.882735:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767920.650633:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767920.906738:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767921.674709:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767921.930745:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767922.698638:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767922.954711:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1775767923.722659:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1775767923.722661:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:2.0:1775767923.722669:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1775767923.722673:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767923.722681:0:8:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5301
00000100:00080000:2.0:1775767923.722692:0:8:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767923.722698:0:8:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775767923.722737:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3b80 x1862022114800384/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767923.722761:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767923.722764:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775767923.722765:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767923.722768:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5301
00000100:00080000:2.0:1775767923.722770:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767923.722773:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767923.722788:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775767923.722797:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2d80 x1862022114800512/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767923.722805:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767923.722808:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00100000:3.0:1775767923.722808:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114800384:192.168.202.110@tcp:250:kworker.0
00000100:00080000:2.0:1775767923.722809:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767923.722818:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5301
00000100:00080000:2.0:1775767923.722821:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767923.722825:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767923.722830:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775767923.722843:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1500 x1862022114800640/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767923.722889:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000800:00000100:3.0:1775767923.722894:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:2.0:1775767923.722895:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775767923.722899:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767923.722903:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5301
00000100:00080000:2.0:1775767923.722907:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767923.722910:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767923.722915:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775767923.722925:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2a00 x1862022114800768/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767923.722932:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767923.722936:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00100000:3.0:1775767923.722936:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767923/real 1775767923]  req@ffff9f14852d3b80 x1862022114800384/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767948 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767923.722940:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767923.722946:0:8:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5301
00000100:00080000:2.0:1775767923.722951:0:8:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767923.722956:0:8:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767923.722960:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775767923.722965:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3b80 x1862022114800384/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767948 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767923.722968:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0380 x1862022114800896/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767923.722985:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767923.722986:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767923.722991:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775767923.722993:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767923.722995:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:2.0:1775767923.722998:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5301
00000100:00080000:2.0:1775767923.723001:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:3.0:1775767923.723001:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114800384:192.168.202.110@tcp:250:kworker.0
00000100:00080000:2.0:1775767923.723006:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767923.723009:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775767923.723015:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0700 x1862022114801024/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767923.723023:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:3.0:1775767923.723027:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114800512:192.168.202.110@tcp:38:kworker.0
00000100:00080000:2.0:1775767923.723028:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775767923.723031:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767923.723036:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5301
00000100:00080000:2.0:1775767923.723039:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767923.723042:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767923.723045:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000800:00000100:3.0:1775767923.723053:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767923.723054:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0a80 x1862022114801152/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767923.723061:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767923.723064:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775767923.723066:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:3.0:1775767923.723067:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767923/real 1775767923]  req@ffff9f14852d2d80 x1862022114800512/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767948 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767923.723074:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5301
00000100:00080000:2.0:1775767923.723080:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767923.723085:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775767923.723089:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2d80 x1862022114800512/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767948 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:2.0:1775767923.723090:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00080000:3.0:1775767923.723100:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00100000:2.0:1775767923.723101:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3800 x1862022114801280/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767923.723109:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767923.723116:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114800512:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767923.723123:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114800640:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775767923.723137:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767923.723143:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767923/real 1775767923]  req@ffff9f14852d1500 x1862022114800640/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767948 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767923.723151:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1500 x1862022114800640/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767948 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767923.723157:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767923.723159:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767923.723161:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114800640:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767923.723165:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114800768:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775767923.723175:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767923.723180:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767923/real 1775767923]  req@ffff9f14852d2a00 x1862022114800768/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767948 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767923.723187:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2a00 x1862022114800768/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767948 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767923.723192:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767923.723194:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767923.723196:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114800768:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775767923.723199:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114800896:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775767923.723208:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767923.723214:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767923/real 1775767923]  req@ffff9f14852d0380 x1862022114800896/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767948 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767923.723221:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0380 x1862022114800896/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767948 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767923.723226:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767923.723228:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767923.723229:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114800896:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775767923.723241:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114801024:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775767923.723250:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767923.723256:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767923/real 1775767923]  req@ffff9f14852d0700 x1862022114801024/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767948 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767923.723263:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0700 x1862022114801024/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767948 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767923.723268:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767923.723270:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767923.723272:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114801024:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767923.723275:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114801152:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775767923.723284:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767923.723289:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767923/real 1775767923]  req@ffff9f14852d0a80 x1862022114801152/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767948 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767923.723296:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0a80 x1862022114801152/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767948 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767923.723306:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767923.723308:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767923.723310:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114801152:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767923.723313:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114801280:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775767923.723321:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767923.723326:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767923/real 1775767923]  req@ffff9f14852d3800 x1862022114801280/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767948 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767923.723341:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3800 x1862022114801280/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767948 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767923.723346:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767923.723348:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767923.723350:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114801280:192.168.202.110@tcp:8:kworker.0
00010000:00010000:3.0:1775767923.978726:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767924.746637:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767925.002708:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767925.770652:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767926.026738:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767926.794648:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767927.050767:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767927.818690:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767928.074771:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1775767928.842669:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767928.842677:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1775767928.842682:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00100000:3.0:1775767928.842683:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:0.0:1775767928.842693:0:8:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5306
00000100:00080000:0.0:1775767928.842703:0:8:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767928.842709:0:8:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775767928.842748:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0000 x1862022114801408/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767928.842770:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767928.842773:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775767928.842774:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767928.842777:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5306
00000100:00080000:0.0:1775767928.842780:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767928.842782:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767928.842792:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767928.842801:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3800 x1862022114801536/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767928.842823:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767928.842827:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00100000:3.0:1775767928.842827:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114801408:192.168.202.110@tcp:250:kworker.0
00000100:00080000:0.0:1775767928.842829:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767928.842836:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5306
00000100:00080000:0.0:1775767928.842839:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767928.842842:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767928.842846:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767928.842856:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d2a00 x1862022114801664/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767928.842863:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767928.842865:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775767928.842866:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767928.842869:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5306
00000100:00080000:0.0:1775767928.842873:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767928.842875:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767928.842879:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775767928.842886:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d2300 x1862022114801792/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767928.842892:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767928.842894:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1775767928.842895:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767928.842898:0:8:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5306
00000100:00080000:0.0:1775767928.842900:0:8:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767928.842902:0:8:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767928.842905:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775767928.842911:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0a80 x1862022114801920/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767928.842917:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767928.842919:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775767928.842921:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767928.842925:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5306
00000100:00080000:0.0:1775767928.842931:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767928.842934:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767928.842938:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767928.842941:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114801536:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767928.842948:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d1500 x1862022114802048/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767928.842956:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775767928.842958:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:0.0:1775767928.842960:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767928.842963:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5306
00000100:00080000:0.0:1775767928.842965:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767928.842967:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767928.842970:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767928.842987:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0e00 x1862022114802176/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767928.842994:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:3.0:1775767928.842995:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114801664:192.168.202.110@tcp:38:kworker.0
00000100:00080000:0.0:1775767928.843000:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775767928.843004:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767928.843009:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5306
00000100:00080000:0.0:1775767928.843014:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775767928.843019:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775767928.843026:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775767928.843030:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114801792:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775767928.843042:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3100 x1862022114802304/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775767928.843059:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114801920:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775767928.843083:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114802048:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767928.843097:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114802176:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767928.843110:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114802304:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775767929.098696:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775767929.866652:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767930.122749:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775767930.890658:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767931.146726:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775767931.914676:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000400:00000100:1.0:1775767931.914904:0:2393:0:(lib-socket.c:454:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.202.110:988
00000400:00000100:1.0:1775767931.914950:0:2393:0:(acceptor.c:111:lnet_connect_console_error()) Connection to 192.168.202.110@tcp at host 192.168.202.110 was unreachable: the network or that node may be down, or Lustre may be misconfigured.
00000800:00000100:1.0:1775767931.914967:0:2393:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:1.0:1775767931.915008:0:2393:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:1.0:1775767931.915018:0:2393:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:1.0:1775767931.915028:0:2393:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:1.0:1775767931.915049:0:2393:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:1.0:1775767931.915064:0:2393:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000100:00100000:3.0:1775767931.915073:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767928/real 1775767931]  req@ffff9f14a04d3800 x1862022114801536/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767953 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000800:00000100:1.0:1775767931.915082:0:2393:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:1.0:1775767931.915097:0:2393:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000100:00100000:3.0:1775767931.915144:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3800 x1862022114801536/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767953 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767931.915157:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767931.915163:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767931.915167:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114801536:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767931.915176:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767928/real 1775767931]  req@ffff9f14a04d2a00 x1862022114801664/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767953 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767931.915189:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d2a00 x1862022114801664/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767953 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767931.915197:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767931.915202:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767931.915205:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114801664:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767931.915213:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767928/real 1775767931]  req@ffff9f14a04d2300 x1862022114801792/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767953 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767931.915226:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d2300 x1862022114801792/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767953 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767931.915234:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767931.915238:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767931.915241:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114801792:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775767931.915247:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767928/real 1775767931]  req@ffff9f14a04d0a80 x1862022114801920/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767953 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767931.915258:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0a80 x1862022114801920/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767953 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767931.915265:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767931.915268:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767931.915272:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114801920:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775767931.915279:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767928/real 1775767931]  req@ffff9f14a04d0000 x1862022114801408/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767953 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767931.915292:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0000 x1862022114801408/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767953 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767931.915299:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767931.915303:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767931.915306:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114801408:192.168.202.110@tcp:250:kworker.0
00000100:00100000:3.0:1775767931.915345:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767928/real 1775767931]  req@ffff9f14a04d1500 x1862022114802048/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767953 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767931.915358:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1500 x1862022114802048/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767953 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767931.915373:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767931.915376:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767931.915380:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114802048:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767931.915389:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767928/real 1775767931]  req@ffff9f14a04d0e00 x1862022114802176/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767953 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767931.915401:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0e00 x1862022114802176/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767953 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767931.915408:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767931.915412:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767931.915415:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114802176:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767931.915423:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767928/real 1775767931]  req@ffff9f14a04d3100 x1862022114802304/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767953 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767931.915435:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3100 x1862022114802304/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767953 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767931.915442:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767931.915445:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767931.915448:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114802304:192.168.202.110@tcp:8:kworker.0
00010000:00010000:3.0:1775767932.170832:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775767932.938628:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767933.194699:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775767933.962628:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:1.0:1775767933.962676:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767933.962681:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1775767933.962685:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767933.962694:0:8:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5311
00000100:00080000:1.0:1775767933.962703:0:8:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767933.962708:0:8:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775767933.962749:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fd880 x1862022114802432/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767933.962789:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767933.962793:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775767933.962795:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767933.962799:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5311
00000100:00080000:1.0:1775767933.962804:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767933.962808:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767933.962821:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767933.962838:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fe300 x1862022114802560/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775767933.962842:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114802432:192.168.202.110@tcp:250:kworker.0
00000100:00080000:1.0:1775767933.962859:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767933.962862:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775767933.962864:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767933.962870:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5311
00000100:00080000:1.0:1775767933.962874:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767933.962879:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767933.962885:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767933.962898:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdf80 x1862022114802688/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767933.962909:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767933.962913:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775767933.962916:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000800:00000100:3.0:1775767933.962919:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:1.0:1775767933.962923:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5311
00000100:00080000:1.0:1775767933.962931:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767933.962936:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767933.962943:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775767933.962956:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fea00 x1862022114802816/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767933.962968:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:3.0:1775767933.962969:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767933/real 1775767933]  req@ffff9f14ab4fd880 x1862022114802432/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767958 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767933.962976:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1775767933.962981:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767933.962990:0:8:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5311
00000100:00080000:1.0:1775767933.962995:0:8:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767933.963000:0:8:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775767933.963003:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fd880 x1862022114802432/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767958 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:1.0:1775767933.963009:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00080000:3.0:1775767933.963022:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00100000:1.0:1775767933.963030:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdc00 x1862022114802944/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767933.963030:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767933.963038:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114802432:192.168.202.110@tcp:250:kworker.0
00000100:00080000:1.0:1775767933.963054:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767933.963059:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775767933.963061:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767933.963067:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5311
00000100:00080000:1.0:1775767933.963071:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:3.0:1775767933.963075:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114802560:192.168.202.110@tcp:38:kworker.0
00000100:00080000:1.0:1775767933.963090:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767933.963097:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000800:00000100:3.0:1775767933.963107:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767933.963114:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fce00 x1862022114803072/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767933.963130:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:3.0:1775767933.963131:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767933/real 1775767933]  req@ffff9f14ab4fe300 x1862022114802560/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767958 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767933.963136:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775767933.963141:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767933.963153:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5311
00000100:00080000:1.0:1775767933.963158:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:3.0:1775767933.963161:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fe300 x1862022114802560/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767958 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767933.963165:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767933.963177:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00080000:3.0:1775767933.963178:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767933.963187:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767933.963193:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114802560:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767933.963204:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114802688:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775767933.963207:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff100 x1862022114803200/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767933.963220:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767933.963223:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775767933.963225:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767933.963230:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5311
00000100:00080000:1.0:1775767933.963234:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767933.963239:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767933.963245:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:3.0:1775767933.963249:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767933.963263:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fc380 x1862022114803328/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775767933.963269:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767933/real 1775767933]  req@ffff9f14ab4fdf80 x1862022114802688/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767958 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767933.963295:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdf80 x1862022114802688/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767958 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767933.963304:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767933.963308:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767933.963312:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114802688:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767933.963318:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114802816:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775767933.963515:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767933.963527:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767933/real 1775767933]  req@ffff9f14ab4fea00 x1862022114802816/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767958 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767933.963539:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fea00 x1862022114802816/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767958 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767933.963546:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767933.963549:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767933.963578:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114802816:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775767933.963585:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114802944:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775767933.963607:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767933.963617:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767933/real 1775767933]  req@ffff9f14ab4fdc00 x1862022114802944/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767958 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767933.963629:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdc00 x1862022114802944/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767958 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767933.963636:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767933.963639:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767933.963642:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114802944:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775767933.963661:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114803072:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775767933.963684:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767933.963695:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767933/real 1775767933]  req@ffff9f14ab4fce00 x1862022114803072/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767958 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767933.963726:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fce00 x1862022114803072/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767958 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767933.963735:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767933.963739:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767933.963743:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114803072:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767933.963749:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114803200:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775767933.963785:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767933.963797:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767933/real 1775767933]  req@ffff9f14ab4ff100 x1862022114803200/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767958 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767933.963809:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff100 x1862022114803200/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767958 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767933.963817:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767933.963821:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767933.963824:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114803200:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767933.963831:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114803328:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775767933.963852:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767933.963863:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767933/real 1775767933]  req@ffff9f14ab4fc380 x1862022114803328/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767958 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767933.963876:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fc380 x1862022114803328/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767958 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767933.963884:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767933.963887:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767933.963891:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114803328:192.168.202.110@tcp:8:kworker.0
00010000:00010000:3.0:1775767934.218731:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775767934.986628:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767935.242704:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775767936.010629:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767936.266800:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767937.034668:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767937.290710:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767938.058662:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767938.314701:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767939.082673:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775767939.082675:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767939.082683:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775767939.082686:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767939.082695:0:8:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5316
00000100:00080000:3.0:1775767939.082704:0:8:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767939.082726:0:8:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775767939.082762:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744700 x1862022114803456/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767939.082784:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767939.082786:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775767939.082788:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767939.082791:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5316
00000100:00080000:3.0:1775767939.082795:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767939.082799:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767939.082810:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767939.082826:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745f80 x1862022114803584/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:1.0:1775767939.082833:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114803456:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775767939.082845:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767939.082849:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775767939.082851:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767939.082857:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5316
00000100:00080000:3.0:1775767939.082862:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767939.082866:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767939.082872:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767939.082885:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747800 x1862022114803712/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767939.082894:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767939.082897:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775767939.082898:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000800:00000100:1.0:1775767939.082904:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775767939.082905:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5316
00000100:00080000:3.0:1775767939.082910:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767939.082914:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767939.082919:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775767939.082929:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744000 x1862022114803840/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767939.082935:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767939.082938:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775767939.082939:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767939.082941:0:8:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5316
00000100:00080000:3.0:1775767939.082944:0:8:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767939.082947:0:8:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767939.082950:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775767939.082952:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767939/real 1775767939]  req@ffff9f1485744700 x1862022114803456/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767964 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767939.082960:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747100 x1862022114803968/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767939.082974:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767939.082977:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775767939.082979:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:1.0:1775767939.082983:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744700 x1862022114803456/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767964 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767939.082984:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5316
00000100:00080000:3.0:1775767939.082989:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767939.082995:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767939.083002:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00080000:1.0:1775767939.083007:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00100000:3.0:1775767939.083012:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745180 x1862022114804096/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767939.083016:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767939.083025:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114803456:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775767939.083029:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767939.083034:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775767939.083036:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767939.083042:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5316
00000100:00080000:3.0:1775767939.083045:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767939.083050:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775767939.083052:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114803584:192.168.202.110@tcp:38:kworker.0
00000008:00100000:3.0:1775767939.083058:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767939.083075:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745c00 x1862022114804224/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:1.0:1775767939.083087:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775767939.083089:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767939.083094:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775767939.083096:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767939.083100:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5316
00000100:00080000:3.0:1775767939.083104:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:1.0:1775767939.083107:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767939/real 1775767939]  req@ffff9f1485745f80 x1862022114803584/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767964 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767939.083109:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767939.083115:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775767939.083129:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746680 x1862022114804352/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:1.0:1775767939.083134:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745f80 x1862022114803584/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767964 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767939.083149:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767939.083154:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767939.083159:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114803584:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775767939.083166:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114803712:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775767939.083187:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767939.083196:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767939/real 1775767939]  req@ffff9f1485747800 x1862022114803712/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767964 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767939.083209:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747800 x1862022114803712/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767964 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767939.083218:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767939.083221:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767939.083224:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114803712:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775767939.083230:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114803840:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775767939.083245:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767939.083253:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767939/real 1775767939]  req@ffff9f1485744000 x1862022114803840/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767964 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767939.083264:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744000 x1862022114803840/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767964 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767939.083274:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767939.083277:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767939.083280:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114803840:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775767939.083286:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114803968:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775767939.083298:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767939.083306:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767939/real 1775767939]  req@ffff9f1485747100 x1862022114803968/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767964 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767939.083318:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747100 x1862022114803968/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767964 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767939.083328:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767939.083331:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767939.083334:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114803968:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775767939.083340:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114804096:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775767939.083355:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767939.083365:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767939/real 1775767939]  req@ffff9f1485745180 x1862022114804096/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767964 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767939.083377:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745180 x1862022114804096/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767964 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767939.083386:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767939.083389:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767939.083393:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114804096:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775767939.083413:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114804224:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775767939.083433:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767939.083441:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767939/real 1775767939]  req@ffff9f1485745c00 x1862022114804224/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767964 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767939.083449:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745c00 x1862022114804224/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767964 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767939.083454:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767939.083456:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767939.083458:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114804224:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775767939.083462:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114804352:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775767939.083472:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767939.083477:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767939/real 1775767939]  req@ffff9f1485746680 x1862022114804352/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767964 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767939.083484:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746680 x1862022114804352/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767964 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767939.083498:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767939.083500:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767939.083502:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114804352:192.168.202.110@tcp:8:kworker.0
00010000:00010000:3.0:1775767939.338700:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767940.106687:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767940.362766:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767941.130761:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767941.386771:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767942.154637:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767942.410707:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767943.178629:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767943.434781:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1775767944.202720:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:2.0:1775767944.202726:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:1.0:1775767944.202731:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1775767944.202738:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767944.202751:0:8:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5321
00000100:00080000:1.0:1775767944.202764:0:8:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767944.202772:0:8:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775767944.202810:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff800 x1862022114804480/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767944.202834:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767944.202837:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775767944.202838:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767944.202841:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5321
00000100:00080000:1.0:1775767944.202844:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767944.202846:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767944.202855:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767944.202863:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fed80 x1862022114804608/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767944.202870:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767944.202873:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775767944.202874:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767944.202876:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5321
00000100:00080000:1.0:1775767944.202878:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767944.202880:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767944.202883:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767944.202889:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114804480:192.168.202.110@tcp:250:kworker.0
00000100:00100000:1.0:1775767944.202892:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ffb80 x1862022114804736/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767944.202914:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767944.202916:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775767944.202917:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767944.202919:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5321
00000100:00080000:1.0:1775767944.202921:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767944.202923:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767944.202926:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775767944.202933:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fca80 x1862022114804864/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767944.202939:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767944.202941:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1775767944.202943:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767944.202945:0:8:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5321
00000100:00080000:1.0:1775767944.202947:0:8:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767944.202949:0:8:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767944.202952:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775767944.202959:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fc380 x1862022114804992/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767944.202965:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767944.202968:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775767944.202968:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767944.202971:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5321
00000100:00080000:1.0:1775767944.202974:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767944.202977:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767944.202983:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000800:00000100:3.0:1775767944.202983:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767944.202994:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff100 x1862022114805120/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767944.203000:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767944.203002:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775767944.203003:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767944.203006:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5321
00000100:00080000:1.0:1775767944.203009:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767944.203011:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767944.203013:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767944.203024:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fce00 x1862022114805248/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767944.203033:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767944.203036:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775767944.203038:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767944.203044:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5321
00000100:00100000:3.0:1775767944.203045:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767944/real 1775767944]  req@ffff9f14ab4ff800 x1862022114804480/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767969 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767944.203048:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767944.203057:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767944.203067:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775767944.203080:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff800 x1862022114804480/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767969 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767944.203084:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdc00 x1862022114805376/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767944.203107:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767944.203122:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767944.203129:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114804480:192.168.202.110@tcp:250:kworker.0
00000100:00100000:3.0:1775767944.203138:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114804608:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775767944.203167:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767944.203180:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767944/real 1775767944]  req@ffff9f14ab4fed80 x1862022114804608/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767969 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767944.203194:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fed80 x1862022114804608/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767969 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767944.203203:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767944.203207:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767944.203210:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114804608:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767944.203245:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ffb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114804736:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775767944.203267:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767944.203278:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767944/real 1775767944]  req@ffff9f14ab4ffb80 x1862022114804736/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767969 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767944.203298:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ffb80 x1862022114804736/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767969 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767944.203306:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767944.203310:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767944.203313:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ffb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114804736:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767944.203319:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114804864:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775767944.203350:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767944.203362:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767944/real 1775767944]  req@ffff9f14ab4fca80 x1862022114804864/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767969 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767944.203374:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fca80 x1862022114804864/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767969 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767944.203383:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767944.203386:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767944.203390:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114804864:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775767944.203396:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114804992:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775767944.203417:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767944.203427:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767944/real 1775767944]  req@ffff9f14ab4fc380 x1862022114804992/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767969 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767944.203440:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fc380 x1862022114804992/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767969 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767944.203455:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767944.203459:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767944.203463:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114804992:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775767944.203469:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114805120:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775767944.203488:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767944.203500:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767944/real 1775767944]  req@ffff9f14ab4ff100 x1862022114805120/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767969 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767944.203513:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff100 x1862022114805120/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767969 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767944.203521:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767944.203525:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767944.203528:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114805120:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767944.203534:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114805248:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775767944.203578:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767944.203590:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767944/real 1775767944]  req@ffff9f14ab4fce00 x1862022114805248/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767969 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767944.203603:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fce00 x1862022114805248/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767969 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767944.203611:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767944.203614:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767944.203618:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114805248:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767944.203624:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114805376:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775767944.203646:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767944.203658:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767944/real 1775767944]  req@ffff9f14ab4fdc00 x1862022114805376/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767969 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767944.203686:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdc00 x1862022114805376/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767969 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767944.203695:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767944.203699:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767944.203702:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114805376:192.168.202.110@tcp:8:kworker.0
00010000:00010000:3.0:1775767944.458735:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767945.226752:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767945.482769:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767946.250682:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767946.506740:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767947.274705:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767947.530793:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767948.298669:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767948.554837:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775767949.322739:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:2.0:1775767949.322740:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767949.322755:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1775767949.322760:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767949.322774:0:8:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5326
00000100:00080000:2.0:1775767949.322790:0:8:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767949.322799:0:8:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775767949.322844:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1f80 x1862022114805504/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767949.322880:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767949.322884:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775767949.322886:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767949.322892:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5326
00000100:00080000:2.0:1775767949.322895:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767949.322901:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767949.322913:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767949.322923:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114805504:192.168.202.110@tcp:250:kworker.0
00000100:00100000:2.0:1775767949.322931:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2680 x1862022114805632/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767949.322943:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767949.322947:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775767949.322949:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767949.322954:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5326
00000100:00080000:2.0:1775767949.322959:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767949.322963:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767949.322969:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775767949.322984:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3800 x1862022114805760/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767949.322994:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767949.322998:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775767949.322999:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767949.323005:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5326
00000800:00000100:3.0:1775767949.323011:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:2.0:1775767949.323017:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767949.323024:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767949.323032:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775767949.323047:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0a80 x1862022114805888/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767949.323058:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767949.323061:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1775767949.323067:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:3.0:1775767949.323068:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767949/real 1775767949]  req@ffff9f14852d1f80 x1862022114805504/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767974 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767949.323074:0:8:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5326
00000100:00080000:2.0:1775767949.323083:0:8:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767949.323091:0:8:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767949.323100:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775767949.323111:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1f80 x1862022114805504/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767974 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767949.323116:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0700 x1862022114806016/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767949.323154:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767949.323160:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767949.323165:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:2.0:1775767949.323175:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775767949.323179:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:3.0:1775767949.323180:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114805504:192.168.202.110@tcp:250:kworker.0
00000100:00080000:2.0:1775767949.323190:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5326
00000100:00080000:2.0:1775767949.323195:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767949.323201:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767949.323207:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775767949.323219:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0380 x1862022114806144/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775767949.323223:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114805632:192.168.202.110@tcp:38:kworker.0
00000100:00080000:2.0:1775767949.323251:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767949.323254:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775767949.323256:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767949.323263:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5326
00000800:00000100:3.0:1775767949.323264:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:2.0:1775767949.323272:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767949.323280:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775767949.323286:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767949/real 1775767949]  req@ffff9f14852d2680 x1862022114805632/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767974 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:2.0:1775767949.323287:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767949.323310:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2680 x1862022114805632/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767974 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767949.323314:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2a00 x1862022114806272/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767949.323332:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767949.323345:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00080000:2.0:1775767949.323350:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:3.0:1775767949.323353:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114805632:192.168.202.110@tcp:38:kworker.0
00000100:00080000:2.0:1775767949.323365:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775767949.323369:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:3.0:1775767949.323369:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114805760:192.168.202.110@tcp:38:kworker.0
00000100:00080000:2.0:1775767949.323386:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5326
00000100:00080000:2.0:1775767949.323391:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767949.323395:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767949.323402:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:3.0:1775767949.323409:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767949.323419:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1500 x1862022114806400/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775767949.323423:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767949/real 1775767949]  req@ffff9f14852d3800 x1862022114805760/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767974 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767949.323484:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3800 x1862022114805760/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767974 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767949.323493:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767949.323498:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767949.323500:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114805760:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767949.323508:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114805888:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775767949.323530:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767949.323543:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767949/real 1775767949]  req@ffff9f14852d0a80 x1862022114805888/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767974 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767949.323603:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0a80 x1862022114805888/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767974 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767949.323629:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767949.323635:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767949.323639:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114805888:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775767949.323648:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114806016:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775767949.323678:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767949.323688:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767949/real 1775767949]  req@ffff9f14852d0700 x1862022114806016/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767974 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767949.323698:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0700 x1862022114806016/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767974 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767949.323707:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767949.323713:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767949.323717:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114806016:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775767949.323743:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114806144:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775767949.323781:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767949.323808:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767949/real 1775767949]  req@ffff9f14852d0380 x1862022114806144/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767974 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767949.323836:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0380 x1862022114806144/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767974 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767949.323917:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767949.323925:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767949.323931:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114806144:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767949.324065:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114806272:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775767949.324088:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767949.324143:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767949/real 1775767949]  req@ffff9f14852d2a00 x1862022114806272/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767974 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767949.324166:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2a00 x1862022114806272/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767974 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767949.324178:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767949.324183:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767949.324188:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114806272:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767949.324196:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114806400:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775767949.324218:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767949.324231:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767949/real 1775767949]  req@ffff9f14852d1500 x1862022114806400/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767974 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767949.324242:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1500 x1862022114806400/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767974 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767949.324251:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767949.324254:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767949.324256:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114806400:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775767949.578783:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775767950.346658:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767950.602767:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775767951.370677:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767951.626750:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775767952.394652:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767952.650770:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775767953.418716:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767953.674903:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775767954.442659:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:2.0:1775767954.442679:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767954.442687:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1775767954.442693:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767954.442706:0:8:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5331
00000100:00080000:2.0:1775767954.442719:0:8:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767954.442727:0:8:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775767954.442774:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2d80 x1862022114806528/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767954.442802:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767954.442806:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775767954.442808:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767954.442813:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5331
00000100:00080000:2.0:1775767954.442817:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767954.442822:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767954.442833:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775767954.442850:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3b80 x1862022114806656/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775767954.442850:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114806528:192.168.202.110@tcp:250:kworker.0
00000100:00080000:2.0:1775767954.442867:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767954.442871:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775767954.442874:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767954.442879:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5331
00000100:00080000:2.0:1775767954.442883:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767954.442888:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767954.442894:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775767954.442906:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0000 x1862022114806784/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767954.442923:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767954.442926:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775767954.442928:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767954.442932:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5331
00000100:00080000:2.0:1775767954.442937:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000800:00000100:3.0:1775767954.442938:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:2.0:1775767954.442945:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767954.442951:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775767954.442965:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1180 x1862022114806912/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767954.442975:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767954.442980:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1775767954.442982:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767954.442987:0:8:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5331
00000100:00080000:2.0:1775767954.442991:0:8:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00100000:3.0:1775767954.442994:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767954/real 1775767954]  req@ffff9f14852d2d80 x1862022114806528/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767979 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767954.442999:0:8:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767954.443007:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775767954.443021:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1880 x1862022114807040/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775767954.443028:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2d80 x1862022114806528/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767979 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767954.443036:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767954.443046:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775767954.443049:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767954.443053:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767954.443057:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5331
00000100:00080000:3.0:1775767954.443063:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:2.0:1775767954.443065:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:3.0:1775767954.443070:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114806528:192.168.202.110@tcp:250:kworker.0
00000100:00080000:2.0:1775767954.443075:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767954.443084:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775767954.443096:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0e00 x1862022114807168/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775767954.443105:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114806656:192.168.202.110@tcp:38:kworker.0
00000100:00080000:2.0:1775767954.443110:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767954.443115:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775767954.443117:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767954.443132:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5331
00000800:00000100:3.0:1775767954.443138:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:2.0:1775767954.443140:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767954.443147:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767954.443153:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767954.443158:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767954/real 1775767954]  req@ffff9f14852d3b80 x1862022114806656/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767979 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767954.443170:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2300 x1862022114807296/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775767954.443178:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3b80 x1862022114806656/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767979 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767954.443189:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767954.443196:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767954.443197:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775767954.443200:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767954.443205:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00080000:2.0:1775767954.443210:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5331
00000100:00100000:3.0:1775767954.443214:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114806656:192.168.202.110@tcp:38:kworker.0
00000100:00080000:2.0:1775767954.443217:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767954.443226:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775767954.443226:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114806784:192.168.202.110@tcp:38:kworker.0
00000008:00100000:2.0:1775767954.443236:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775767954.443251:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3100 x1862022114807424/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:3.0:1775767954.443272:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767954.443285:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767954/real 1775767954]  req@ffff9f14852d0000 x1862022114806784/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767979 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767954.443299:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0000 x1862022114806784/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767979 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767954.443308:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767954.443312:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767954.443315:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114806784:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767954.443322:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114806912:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775767954.443344:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767954.443355:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767954/real 1775767954]  req@ffff9f14852d1180 x1862022114806912/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767979 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767954.443368:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1180 x1862022114806912/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767979 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767954.443384:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767954.443387:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767954.443391:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114806912:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775767954.443397:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114807040:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775767954.443419:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767954.443430:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767954/real 1775767954]  req@ffff9f14852d1880 x1862022114807040/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767979 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767954.443442:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1880 x1862022114807040/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767979 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767954.443450:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767954.443453:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767954.443456:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114807040:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775767954.443476:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114807168:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775767954.443497:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767954.443508:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767954/real 1775767954]  req@ffff9f14852d0e00 x1862022114807168/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767979 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767954.443521:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0e00 x1862022114807168/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767979 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767954.443530:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767954.443533:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767954.443538:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114807168:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767954.443544:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114807296:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775767954.443607:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767954.443619:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767954/real 1775767954]  req@ffff9f14852d2300 x1862022114807296/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767979 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767954.443631:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2300 x1862022114807296/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767979 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767954.443640:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767954.443643:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767954.443647:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114807296:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767954.443653:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114807424:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775767954.443673:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767954.443685:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767954/real 1775767954]  req@ffff9f14852d3100 x1862022114807424/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767979 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767954.443698:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3100 x1862022114807424/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767979 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767954.443706:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767954.443710:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767954.443713:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114807424:192.168.202.110@tcp:8:kworker.0
00010000:00010000:3.0:1775767954.698796:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775767955.466619:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767955.722707:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775767956.490677:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767956.746725:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767957.514656:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767957.770723:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767958.538657:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767958.794668:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1775767959.562646:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:3.0:1775767959.562652:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:2.0:1775767959.562653:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1775767959.562658:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767959.562667:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5337
00000100:00080000:2.0:1775767959.562677:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767959.562683:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775767959.562717:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1c00 x1862022114807552/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767959.562741:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767959.562744:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775767959.562745:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767959.562748:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5337
00000100:00080000:2.0:1775767959.562751:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767959.562753:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767959.562761:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775767959.562769:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3480 x1862022114807680/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767959.562775:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767959.562777:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775767959.562779:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767959.562782:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5337
00000100:00080000:2.0:1775767959.562784:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767959.562786:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767959.562789:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775767959.562795:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3100 x1862022114807808/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767959.562801:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767959.562803:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775767959.562805:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767959.562807:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5337
00000100:00080000:2.0:1775767959.562810:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767959.562813:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767959.562816:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775767959.562822:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114807552:192.168.202.110@tcp:250:kworker.0
00000100:00100000:2.0:1775767959.562823:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2300 x1862022114807936/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767959.562835:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767959.562837:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1775767959.562838:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767959.562841:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5337
00000100:00080000:2.0:1775767959.562843:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767959.562845:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767959.562848:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775767959.562856:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0e00 x1862022114808064/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767959.562863:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767959.562866:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775767959.562867:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767959.562869:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5337
00000100:00080000:2.0:1775767959.562871:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767959.562874:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767959.562877:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000800:00000100:3.0:1775767959.562886:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775767959.562887:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1880 x1862022114808192/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767959.562895:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767959.562897:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775767959.562899:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767959.562902:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5337
00000100:00080000:2.0:1775767959.562905:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767959.562907:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767959.562910:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775767959.562917:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1180 x1862022114808320/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767959.562923:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:3.0:1775767959.562926:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767959/real 1775767959]  req@ffff9f14852d1c00 x1862022114807552/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767984 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767959.562927:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775767959.562930:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767959.562938:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5337
00000100:00080000:2.0:1775767959.562943:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767959.562946:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767959.562950:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775767959.562952:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1c00 x1862022114807552/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767984 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767959.562960:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0000 x1862022114808448/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767959.562962:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767959.562970:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767959.562975:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114807552:192.168.202.110@tcp:250:kworker.0
00000100:00100000:3.0:1775767959.562980:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114807680:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775767959.562998:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767959.563006:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767959/real 1775767959]  req@ffff9f14852d3480 x1862022114807680/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767984 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767959.563014:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3480 x1862022114807680/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767984 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767959.563019:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767959.563021:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767959.563032:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114807680:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767959.563036:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114807808:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775767959.563048:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767959.563055:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767959/real 1775767959]  req@ffff9f14852d3100 x1862022114807808/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767984 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767959.563062:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3100 x1862022114807808/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767984 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767959.563068:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767959.563070:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767959.563072:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114807808:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767959.563089:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114807936:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775767959.563119:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767959.563130:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767959/real 1775767959]  req@ffff9f14852d2300 x1862022114807936/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767984 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767959.563142:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2300 x1862022114807936/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767984 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767959.563151:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767959.563155:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767959.563158:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114807936:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775767959.563165:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114808064:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775767959.563186:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767959.563208:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767959/real 1775767959]  req@ffff9f14852d0e00 x1862022114808064/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767984 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767959.563221:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0e00 x1862022114808064/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767984 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767959.563227:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767959.563229:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767959.563232:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114808064:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775767959.563236:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114808192:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775767959.563249:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767959.563256:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767959/real 1775767959]  req@ffff9f14852d1880 x1862022114808192/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767984 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767959.563263:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1880 x1862022114808192/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767984 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767959.563269:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767959.563271:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767959.563273:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114808192:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767959.563276:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114808320:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775767959.563285:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767959.563291:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767959/real 1775767959]  req@ffff9f14852d1180 x1862022114808320/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767984 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767959.563303:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1180 x1862022114808320/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767984 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767959.563313:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767959.563316:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767959.563318:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114808320:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767959.563322:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114808448:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775767959.563333:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767959.563339:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767959/real 1775767959]  req@ffff9f14852d0000 x1862022114808448/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767984 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767959.563346:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0000 x1862022114808448/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767984 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767959.563351:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767959.563353:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775767959.563355:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114808448:192.168.202.110@tcp:8:kworker.0
00010000:00010000:3.0:1775767959.818716:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775767960.586699:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767960.842780:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775767961.610623:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767961.866716:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775767962.634614:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767962.890733:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775767963.658629:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767963.914654:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1775767964.682650:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767964.682660:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00100000:2.0:1775767964.682661:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775767964.682665:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767964.682677:0:8:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5342
00000100:00080000:3.0:1775767964.682689:0:8:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767964.682697:0:8:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775767964.682744:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745500 x1862022114808576/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767964.682769:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767964.682771:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775767964.682773:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767964.682776:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5342
00000100:00080000:3.0:1775767964.682779:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767964.682782:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767964.682790:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767964.682801:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744e00 x1862022114808704/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767964.682808:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:1.0:1775767964.682811:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114808576:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775767964.682812:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775767964.682817:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767964.682823:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5342
00000100:00080000:3.0:1775767964.682826:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767964.682831:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767964.682836:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767964.682851:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744380 x1862022114808832/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767964.682866:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767964.682869:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000800:00000100:1.0:1775767964.682873:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775767964.682874:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767964.682880:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5342
00000100:00080000:3.0:1775767964.682885:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767964.682890:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767964.682896:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775767964.682911:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746a00 x1862022114808960/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:1.0:1775767964.682912:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767964/real 1775767964]  req@ffff9f1485745500 x1862022114808576/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767989 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767964.682944:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:1.0:1775767964.682949:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745500 x1862022114808576/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767989 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767964.682950:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775767964.682954:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767964.682964:0:8:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5342
00000100:00080000:1.0:1775767964.682966:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767964.682968:0:8:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767964.682972:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:3.0:1775767964.682976:0:8:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775767964.682978:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114808576:192.168.202.110@tcp:250:kworker.0
00000008:00100000:3.0:1775767964.682987:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775767964.683002:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114808704:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767964.683003:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744a80 x1862022114809088/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767964.683019:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000800:00000100:1.0:1775767964.683023:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775767964.683027:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775767964.683030:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767964.683036:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5342
00000100:00100000:1.0:1775767964.683041:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767964/real 1775767964]  req@ffff9f1485744e00 x1862022114808704/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767989 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767964.683044:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767964.683053:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767964.683061:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767964.683064:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744e00 x1862022114808704/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767989 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767964.683075:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00100000:3.0:1775767964.683076:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747480 x1862022114809216/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767964.683083:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767964.683092:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114808704:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775767964.683098:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:1.0:1775767964.683103:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114808832:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775767964.683104:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775767964.683109:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767964.683115:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5342
00000100:00080000:3.0:1775767964.683120:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767964.683124:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000800:00000100:1.0:1775767964.683129:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000008:00100000:3.0:1775767964.683134:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767964.683141:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767964/real 1775767964]  req@ffff9f1485744380 x1862022114808832/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767989 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767964.683149:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745880 x1862022114809344/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:1.0:1775767964.683158:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744380 x1862022114808832/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767989 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767964.683163:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767964.683171:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775767964.683178:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767964.683181:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767964.683187:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114808832:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775767964.683190:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:1.0:1775767964.683195:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114808960:192.168.202.110@tcp:8:kworker.0
00000100:00080000:3.0:1775767964.683199:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5342
00000100:00080000:3.0:1775767964.683206:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767964.683211:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767964.683217:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775767964.683230:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746300 x1862022114809472/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:1.0:1775767964.683237:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767964.683251:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767964/real 1775767964]  req@ffff9f1485746a00 x1862022114808960/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767989 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767964.683268:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746a00 x1862022114808960/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767989 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767964.683275:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767964.683277:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767964.683280:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114808960:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775767964.683293:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114809088:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775767964.683305:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767964.683311:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767964/real 1775767964]  req@ffff9f1485744a80 x1862022114809088/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767989 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767964.683318:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744a80 x1862022114809088/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767989 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767964.683324:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767964.683325:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767964.683327:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114809088:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775767964.683332:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114809216:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775767964.683349:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767964.683359:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767964/real 1775767964]  req@ffff9f1485747480 x1862022114809216/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767989 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767964.683371:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747480 x1862022114809216/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767989 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767964.683381:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767964.683384:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767964.683387:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114809216:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775767964.683394:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114809344:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775767964.683414:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767964.683424:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767964/real 1775767964]  req@ffff9f1485745880 x1862022114809344/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767989 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767964.683437:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745880 x1862022114809344/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767989 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767964.683446:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767964.683449:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767964.683453:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114809344:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775767964.683460:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114809472:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775767964.683499:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767964.683507:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767964/real 1775767964]  req@ffff9f1485746300 x1862022114809472/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767989 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767964.683514:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746300 x1862022114809472/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767989 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767964.683520:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767964.683522:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767964.683524:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114809472:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775767964.938665:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767965.706658:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767965.962678:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767966.730662:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767966.986714:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767967.754644:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767968.010675:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775767968.778674:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767969.034704:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1775767969.802626:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:2.0:1775767969.802630:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775767969.802637:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775767969.802641:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767969.802650:0:8:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5347
00000100:00080000:3.0:1775767969.802659:0:8:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767969.802665:0:8:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775767969.802701:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747b80 x1862022114809600/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767969.802722:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767969.802725:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775767969.802726:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767969.802729:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5347
00000100:00080000:3.0:1775767969.802731:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767969.802733:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767969.802742:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767969.802751:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746300 x1862022114809728/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767969.802758:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767969.802761:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775767969.802762:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767969.802765:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5347
00000100:00080000:3.0:1775767969.802767:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767969.802770:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775767969.802774:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114809600:192.168.202.110@tcp:250:kworker.0
00000008:00100000:3.0:1775767969.802775:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767969.802784:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745880 x1862022114809856/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767969.802790:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767969.802792:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775767969.802793:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767969.802796:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5347
00000100:00080000:3.0:1775767969.802798:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767969.802800:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767969.802803:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775767969.802810:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747480 x1862022114809984/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767969.802816:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767969.802818:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775767969.802819:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767969.802821:0:8:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5347
00000100:00080000:3.0:1775767969.802823:0:8:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767969.802826:0:8:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767969.802829:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775767969.802837:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744a80 x1862022114810112/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:1.0:1775767969.802840:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775767969.802847:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767969.802850:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775767969.802851:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767969.802855:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5347
00000100:00080000:3.0:1775767969.802859:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767969.802864:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767969.802870:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767969.802885:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767969/real 1775767969]  req@ffff9f1485747b80 x1862022114809600/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767994 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767969.802886:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746a00 x1862022114810240/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767969.802921:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767969.802923:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775767969.802925:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767969.802928:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5347
00000100:00100000:1.0:1775767969.802931:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747b80 x1862022114809600/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767994 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767969.802933:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767969.802940:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00080000:1.0:1775767969.802946:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000008:00100000:3.0:1775767969.802947:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00080000:1.0:1775767969.802953:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767969.802958:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114809600:192.168.202.110@tcp:250:kworker.0
00000100:00100000:3.0:1775767969.802960:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744380 x1862022114810368/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:1.0:1775767969.802969:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114809728:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775767969.802971:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767969.802976:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775767969.802977:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767969.802982:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5347
00000100:00080000:3.0:1775767969.802984:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767969.802987:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767969.802990:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:1.0:1775767969.802995:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775767969.803000:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744e00 x1862022114810496/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:1.0:1775767969.803008:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767969/real 1775767969]  req@ffff9f1485746300 x1862022114809728/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767994 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767969.803018:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746300 x1862022114809728/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767994 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767969.803024:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767969.803033:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767969.803035:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114809728:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775767969.803054:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114809856:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775767969.803074:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767969.803085:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767969/real 1775767969]  req@ffff9f1485745880 x1862022114809856/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767994 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767969.803098:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745880 x1862022114809856/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767994 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767969.803107:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767969.803111:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767969.803114:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114809856:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775767969.803121:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114809984:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775767969.803138:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767969.803145:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767969/real 1775767969]  req@ffff9f1485747480 x1862022114809984/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767994 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767969.803153:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747480 x1862022114809984/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767994 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767969.803158:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767969.803161:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767969.803163:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114809984:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775767969.803166:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114810112:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775767969.803176:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767969.803181:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767969/real 1775767969]  req@ffff9f1485744a80 x1862022114810112/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767994 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767969.803188:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744a80 x1862022114810112/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767994 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767969.803193:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767969.803195:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767969.803197:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114810112:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775767969.803200:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114810240:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775767969.803209:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767969.803214:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767969/real 1775767969]  req@ffff9f1485746a00 x1862022114810240/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767994 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767969.803221:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746a00 x1862022114810240/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767994 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767969.803226:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767969.803228:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767969.803230:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114810240:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775767969.803234:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114810368:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775767969.803242:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767969.803248:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767969/real 1775767969]  req@ffff9f1485744380 x1862022114810368/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767994 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767969.803254:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744380 x1862022114810368/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767994 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767969.803259:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767969.803261:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767969.803263:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114810368:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775767969.803267:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114810496:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775767969.803283:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767969.803293:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767969/real 1775767969]  req@ffff9f1485744e00 x1862022114810496/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767994 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767969.803305:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744e00 x1862022114810496/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767994 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767969.803314:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767969.803318:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767969.803321:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114810496:192.168.202.110@tcp:8:kworker.0
00010000:00010000:3.0:1775767970.058723:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767970.826705:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767971.082738:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767971.850675:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767972.106755:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767972.874675:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767973.130721:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767973.898654:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767974.154744:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1775767974.922666:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767974.922676:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00100000:0.0:1775767974.922677:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775767974.922681:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767974.922692:0:8:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5352
00000100:00080000:3.0:1775767974.922707:0:8:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767974.922714:0:8:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775767974.922753:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745500 x1862022114810624/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767974.922782:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767974.922785:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775767974.922786:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767974.922790:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5352
00000100:00080000:3.0:1775767974.922792:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767974.922794:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767974.922804:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767974.922817:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746680 x1862022114810752/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767974.922824:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767974.922827:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775767974.922829:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767974.922832:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5352
00000100:00080000:3.0:1775767974.922834:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767974.922839:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775767974.922842:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114810624:192.168.202.110@tcp:250:kworker.0
00000008:00100000:3.0:1775767974.922844:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767974.922860:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745c00 x1862022114810880/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767974.922868:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767974.922871:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775767974.922873:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767974.922877:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5352
00000100:00080000:3.0:1775767974.922881:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767974.922885:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767974.922891:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775767974.922904:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745180 x1862022114811008/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:1.0:1775767974.922915:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775767974.922916:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767974.922923:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775767974.922926:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767974.922934:0:8:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5352
00000100:00080000:3.0:1775767974.922940:0:8:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767974.922944:0:8:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767974.922953:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775767974.922960:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767974/real 1775767974]  req@ffff9f1485745500 x1862022114810624/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767999 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767974.922967:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747100 x1862022114811136/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767974.922992:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:1.0:1775767974.922998:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745500 x1862022114810624/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775767999 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767974.923004:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775767974.923014:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767974.923026:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767974.923030:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5352
00000100:00080000:1.0:1775767974.923036:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:3.0:1775767974.923038:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767974.923048:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775767974.923053:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114810624:192.168.202.110@tcp:250:kworker.0
00000008:00100000:3.0:1775767974.923057:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767974.923069:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114810752:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767974.923076:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744000 x1862022114811264/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767974.923089:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767974.923093:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775767974.923095:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767974.923100:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5352
00000800:00000100:1.0:1775767974.923108:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775767974.923110:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767974.923118:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767974.923124:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767974.923125:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767974/real 1775767974]  req@ffff9f1485746680 x1862022114810752/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767999 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767974.923144:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747800 x1862022114811392/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:1.0:1775767974.923164:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746680 x1862022114810752/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767999 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767974.923170:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767974.923178:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767974.923178:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775767974.923184:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767974.923191:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00080000:3.0:1775767974.923196:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5352
00000100:00100000:1.0:1775767974.923201:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114810752:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775767974.923202:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767974.923211:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767974.923225:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775767974.923231:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114810880:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767974.923239:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745f80 x1862022114811520/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:1.0:1775767974.923252:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767974.923267:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767974/real 1775767974]  req@ffff9f1485745c00 x1862022114810880/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767999 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767974.923279:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745c00 x1862022114810880/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767999 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767974.923301:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767974.923304:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767974.923306:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114810880:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775767974.923311:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114811008:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775767974.923322:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767974.923328:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767974/real 1775767974]  req@ffff9f1485745180 x1862022114811008/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767999 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767974.923335:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745180 x1862022114811008/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767999 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767974.923343:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767974.923346:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767974.923349:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114811008:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775767974.923356:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114811136:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775767974.923490:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767974.923503:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767974/real 1775767974]  req@ffff9f1485747100 x1862022114811136/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767999 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767974.923516:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747100 x1862022114811136/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767999 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767974.923524:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767974.923537:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767974.923542:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114811136:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775767974.923549:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114811264:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775767974.923608:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767974.923623:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767974/real 1775767974]  req@ffff9f1485744000 x1862022114811264/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767999 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767974.923636:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744000 x1862022114811264/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767999 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767974.923645:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767974.923649:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767974.923652:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114811264:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775767974.923659:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114811392:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775767974.923680:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767974.923692:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767974/real 1775767974]  req@ffff9f1485747800 x1862022114811392/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767999 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767974.923705:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747800 x1862022114811392/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775767999 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767974.923713:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767974.923716:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767974.923719:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114811392:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775767974.923740:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114811520:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775767974.923762:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767974.923774:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767974/real 1775767974]  req@ffff9f1485745f80 x1862022114811520/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767999 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767974.923787:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745f80 x1862022114811520/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775767999 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767974.923795:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767974.923799:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767974.923803:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114811520:192.168.202.110@tcp:8:kworker.0
00010000:00010000:3.0:1775767975.178746:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767975.946626:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767976.202731:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767976.970652:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767977.226755:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767977.994648:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767978.250694:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767979.018649:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767979.274759:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1775767980.042662:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1775767980.042668:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775767980.042670:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775767980.042675:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767980.042692:0:8:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5357
00000100:00080000:3.0:1775767980.042703:0:8:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767980.042709:0:8:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775767980.042745:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744700 x1862022114811648/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767980.042771:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767980.042774:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775767980.042775:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767980.042778:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5357
00000100:00080000:3.0:1775767980.042781:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767980.042783:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767980.042791:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767980.042801:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745f80 x1862022114811776/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767980.042808:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767980.042811:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775767980.042812:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767980.042815:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5357
00000100:00080000:3.0:1775767980.042816:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00100000:1.0:1775767980.042821:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114811648:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775767980.042821:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767980.042828:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775767980.042835:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747800 x1862022114811904/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767980.042842:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767980.042844:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775767980.042845:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767980.042848:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5357
00000100:00080000:3.0:1775767980.042850:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767980.042852:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767980.042856:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775767980.042863:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744000 x1862022114812032/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767980.042869:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767980.042871:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775767980.042872:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767980.042875:0:8:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5357
00000100:00080000:3.0:1775767980.042877:0:8:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000800:00000100:1.0:1775767980.042881:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775767980.042883:0:8:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767980.042888:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775767980.042895:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747100 x1862022114812160/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775767980.042901:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767980.042903:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775767980.042904:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775767980.042907:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5357
00000100:00080000:3.0:1775767980.042909:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767980.042911:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767980.042915:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767980.042919:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767980/real 1775767980]  req@ffff9f1485744700 x1862022114811648/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768005 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767980.042933:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745180 x1862022114812288/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:1.0:1775767980.042934:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744700 x1862022114811648/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768005 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767980.042967:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767980.042968:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767980.042974:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775767980.042975:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:3.0:1775767980.042977:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:1.0:1775767980.042981:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114811648:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775767980.042986:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5357
00000100:00100000:1.0:1775767980.042990:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114811776:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775767980.042994:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767980.043000:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767980.043007:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000800:00000100:1.0:1775767980.043010:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767980.043021:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767980/real 1775767980]  req@ffff9f1485745f80 x1862022114811776/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768005 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775767980.043027:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745c00 x1862022114812416/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:1.0:1775767980.043037:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745f80 x1862022114811776/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768005 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775767980.043045:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775767980.043055:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775767980.043059:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767980.043065:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775767980.043069:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5357
00000100:00080000:1.0:1775767980.043072:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767980.043077:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114811776:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775767980.043077:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775767980.043086:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775767980.043093:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775767980.043102:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114811904:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775767980.043107:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746680 x1862022114812544/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:1.0:1775767980.043120:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767980.043129:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767980/real 1775767980]  req@ffff9f1485747800 x1862022114811904/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768005 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767980.043137:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747800 x1862022114811904/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768005 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767980.043143:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767980.043145:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767980.043147:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114811904:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775767980.043151:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114812032:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775767980.043162:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767980.043167:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767980/real 1775767980]  req@ffff9f1485744000 x1862022114812032/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768005 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767980.043174:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744000 x1862022114812032/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768005 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767980.043179:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767980.043180:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767980.043183:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114812032:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775767980.043187:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114812160:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775767980.043196:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767980.043201:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767980/real 1775767980]  req@ffff9f1485747100 x1862022114812160/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768005 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767980.043208:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747100 x1862022114812160/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768005 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767980.043214:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767980.043215:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767980.043217:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114812160:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775767980.043220:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114812288:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775767980.043229:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767980.043234:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767980/real 1775767980]  req@ffff9f1485745180 x1862022114812288/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768005 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767980.043241:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745180 x1862022114812288/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768005 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767980.043246:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767980.043248:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767980.043249:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114812288:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775767980.043253:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114812416:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775767980.043261:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767980.043267:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767980/real 1775767980]  req@ffff9f1485745c00 x1862022114812416/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768005 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767980.043275:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745c00 x1862022114812416/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768005 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767980.043280:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767980.043281:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767980.043283:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114812416:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775767980.043293:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114812544:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775767980.043302:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775767980.043307:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767980/real 1775767980]  req@ffff9f1485746680 x1862022114812544/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768005 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767980.043333:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746680 x1862022114812544/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768005 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767980.043342:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775767980.043346:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775767980.043349:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114812544:192.168.202.110@tcp:8:kworker.0
00010000:00010000:3.0:1775767980.298720:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767981.066630:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767981.322689:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767982.090667:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767982.346724:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767983.114621:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767983.370774:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767984.138650:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767984.394733:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767985.162646:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:1.0:1775767985.162691:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767985.162696:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1775767985.162701:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767985.162712:0:8:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5362
00000100:00080000:1.0:1775767985.162724:0:8:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767985.162731:0:8:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775767985.162777:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fea00 x1862022114812672/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767985.162799:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767985.162802:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775767985.162804:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767985.162807:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5362
00000100:00080000:1.0:1775767985.162812:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767985.162816:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767985.162825:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767985.162835:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdf80 x1862022114812800/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775767985.162840:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114812672:192.168.202.110@tcp:250:kworker.0
00000100:00080000:1.0:1775767985.162845:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767985.162850:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775767985.162852:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767985.162856:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5362
00000100:00080000:1.0:1775767985.162859:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767985.162862:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767985.162865:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767985.162873:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fe300 x1862022114812928/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767985.162882:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767985.162885:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775767985.162888:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767985.162893:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5362
00000100:00080000:1.0:1775767985.162898:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767985.162903:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000800:00000100:0.0:1775767985.162910:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000008:00100000:1.0:1775767985.162910:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775767985.162927:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fd880 x1862022114813056/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775767985.162939:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767985.162942:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1775767985.162944:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767985.162951:0:8:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5362
00000100:00100000:0.0:1775767985.162952:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767985/real 1775767985]  req@ffff9f14ab4fea00 x1862022114812672/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768010 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767985.162956:0:8:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767985.162964:0:8:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767985.162974:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775767985.162983:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fea00 x1862022114812672/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768010 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775767985.162992:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdc00 x1862022114813184/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767985.162999:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767985.163008:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:1.0:1775767985.163008:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767985.163015:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775767985.163018:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775767985.163019:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114812672:192.168.202.110@tcp:250:kworker.0
00000100:00080000:1.0:1775767985.163025:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5362
00000100:00080000:1.0:1775767985.163030:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767985.163033:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767985.163038:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767985.163046:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fce00 x1862022114813312/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775767985.163052:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114812800:192.168.202.110@tcp:38:kworker.0
00000100:00080000:1.0:1775767985.163058:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767985.163061:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775767985.163062:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775767985.163065:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5362
00000100:00080000:1.0:1775767985.163068:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767985.163070:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767985.163074:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775767985.163084:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff100 x1862022114813440/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1775767985.163088:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:1.0:1775767985.163099:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775767985.163102:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775767985.163105:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775767985.163109:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767985/real 1775767985]  req@ffff9f14ab4fdf80 x1862022114812800/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768010 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775767985.163113:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5362
00000100:00080000:1.0:1775767985.163121:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775767985.163129:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775767985.163136:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775767985.163151:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fc380 x1862022114813568/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775767985.163155:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdf80 x1862022114812800/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768010 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767985.163180:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767985.163185:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767985.163191:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114812800:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767985.163198:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114812928:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767985.163223:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767985.163238:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767985/real 1775767985]  req@ffff9f14ab4fe300 x1862022114812928/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768010 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767985.163256:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fe300 x1862022114812928/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768010 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767985.163294:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767985.163301:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767985.163306:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114812928:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767985.163314:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114813056:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767985.163336:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767985.163347:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767985/real 1775767985]  req@ffff9f14ab4fd880 x1862022114813056/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768010 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767985.163359:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fd880 x1862022114813056/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768010 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767985.163374:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767985.163378:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767985.163381:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114813056:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775767985.163387:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114813184:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767985.163403:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767985.163410:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767985/real 1775767985]  req@ffff9f14ab4fdc00 x1862022114813184/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768010 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767985.163418:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdc00 x1862022114813184/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768010 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767985.163423:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767985.163425:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767985.163427:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114813184:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775767985.163439:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114813312:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767985.163449:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767985.163455:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767985/real 1775767985]  req@ffff9f14ab4fce00 x1862022114813312/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768010 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767985.163462:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fce00 x1862022114813312/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768010 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767985.163467:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767985.163469:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767985.163471:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114813312:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767985.163475:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114813440:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767985.163484:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767985.163489:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767985/real 1775767985]  req@ffff9f14ab4ff100 x1862022114813440/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768010 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767985.163496:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff100 x1862022114813440/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768010 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767985.163501:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767985.163502:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767985.163504:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114813440:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767985.163507:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114813568:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767985.163516:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767985.163521:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767985/real 1775767985]  req@ffff9f14ab4fc380 x1862022114813568/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768010 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767985.163527:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fc380 x1862022114813568/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768010 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767985.163533:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767985.163535:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767985.163537:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114813568:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775767985.418692:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767986.186661:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767986.442718:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767987.210648:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767987.466693:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767988.234661:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767988.490720:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767989.258670:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767989.514720:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1775767990.282665:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1775767990.282669:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:2.0:1775767990.282679:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1775767990.282684:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767990.282694:0:8:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5367
00000100:00080000:2.0:1775767990.282705:0:8:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767990.282712:0:8:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775767990.282750:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3b80 x1862022114813696/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767990.282777:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767990.282780:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775767990.282781:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767990.282784:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5367
00000100:00080000:2.0:1775767990.282786:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767990.282789:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767990.282798:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775767990.282806:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2d80 x1862022114813824/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767990.282814:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1775767990.282818:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114813696:192.168.202.110@tcp:250:kworker.0
00000100:00080000:2.0:1775767990.282818:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775767990.282820:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767990.282826:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5367
00000100:00080000:2.0:1775767990.282829:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767990.282831:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767990.282834:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775767990.282842:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1500 x1862022114813952/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767990.282848:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767990.282851:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775767990.282852:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767990.282855:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5367
00000100:00080000:2.0:1775767990.282857:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767990.282860:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767990.282862:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775767990.282869:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2a00 x1862022114814080/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767990.282874:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767990.282877:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1775767990.282878:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767990.282880:0:8:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5367
00000100:00080000:2.0:1775767990.282882:0:8:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767990.282884:0:8:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767990.282888:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775767990.282897:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0380 x1862022114814208/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1775767990.282903:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:2.0:1775767990.282904:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767990.282909:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775767990.282910:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767990.282913:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5367
00000100:00080000:2.0:1775767990.282916:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767990.282919:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767990.282921:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775767990.282929:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0700 x1862022114814336/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767990.282945:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767990.282948:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775767990.282950:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775767990.282954:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767990/real 1775767990]  req@ffff9f14852d3b80 x1862022114813696/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768015 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775767990.282957:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5367
00000100:00080000:2.0:1775767990.282964:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767990.282973:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767990.282980:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767990.282985:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3b80 x1862022114813696/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768015 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775767990.282995:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0a80 x1862022114814464/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775767990.283005:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775767990.283006:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767990.283012:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775767990.283014:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775767990.283016:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:2.0:1775767990.283020:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5367
00000100:00080000:2.0:1775767990.283024:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1775767990.283025:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114813696:192.168.202.110@tcp:250:kworker.0
00000100:00080000:2.0:1775767990.283029:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767990.283036:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775767990.283044:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3800 x1862022114814592/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775767990.283058:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114813824:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767990.283087:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767990.283100:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767990/real 1775767990]  req@ffff9f14852d2d80 x1862022114813824/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768015 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767990.283114:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2d80 x1862022114813824/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768015 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767990.283123:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767990.283127:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767990.283131:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114813824:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767990.283138:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114813952:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767990.283161:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767990.283172:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767990/real 1775767990]  req@ffff9f14852d1500 x1862022114813952/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768015 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767990.283183:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1500 x1862022114813952/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768015 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767990.283192:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767990.283196:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767990.283199:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114813952:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767990.283206:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114814080:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767990.283260:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767990.283272:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767990/real 1775767990]  req@ffff9f14852d2a00 x1862022114814080/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768015 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767990.283285:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2a00 x1862022114814080/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768015 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767990.283294:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767990.283298:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767990.283302:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114814080:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775767990.283308:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114814208:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767990.283330:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767990.283341:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767990/real 1775767990]  req@ffff9f14852d0380 x1862022114814208/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768015 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767990.283362:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0380 x1862022114814208/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768015 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767990.283370:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767990.283374:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767990.283377:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114814208:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775767990.283384:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114814336:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767990.283406:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767990.283418:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767990/real 1775767990]  req@ffff9f14852d0700 x1862022114814336/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768015 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767990.283431:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0700 x1862022114814336/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768015 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767990.283441:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767990.283444:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767990.283448:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114814336:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767990.283454:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114814464:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775767990.283474:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767990.283486:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767990/real 1775767990]  req@ffff9f14852d0a80 x1862022114814464/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768015 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767990.283498:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0a80 x1862022114814464/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768015 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767990.283513:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767990.283516:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767990.283520:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114814464:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767990.283526:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114814592:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775767990.283547:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775767990.283582:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767990/real 1775767990]  req@ffff9f14852d3800 x1862022114814592/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768015 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767990.283596:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3800 x1862022114814592/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768015 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767990.283606:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767990.283609:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767990.283612:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114814592:192.168.202.110@tcp:8:kworker.0
00010000:00010000:3.0:1775767990.538763:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767991.306673:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767991.562749:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767992.330647:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767992.586757:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767993.354626:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767993.610696:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767994.378658:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767994.634673:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767995.402640:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:2.0:1775767995.402640:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767995.402650:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1775767995.402656:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767995.402667:0:8:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5372
00000100:00080000:2.0:1775767995.402679:0:8:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767995.402687:0:8:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775767995.402731:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2680 x1862022114814720/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767995.402759:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767995.402762:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775767995.402764:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767995.402767:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5372
00000100:00080000:2.0:1775767995.402771:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767995.402774:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767995.402784:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775767995.402790:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114814720:192.168.202.110@tcp:250:kworker.0
00000100:00100000:2.0:1775767995.402796:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1f80 x1862022114814848/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767995.402809:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767995.402813:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775767995.402815:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767995.402820:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5372
00000100:00080000:2.0:1775767995.402825:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767995.402829:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767995.402835:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775767995.402849:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3800 x1862022114814976/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767995.402868:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767995.402872:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775767995.402875:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775767995.402876:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114814848:192.168.202.110@tcp:38:kworker.0
00000100:00080000:2.0:1775767995.402883:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5372
00000100:00080000:2.0:1775767995.402889:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767995.402894:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767995.402901:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775767995.402914:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114814976:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775767995.402919:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0a80 x1862022114815104/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767995.402933:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767995.402936:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1775767995.402937:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767995.402940:0:8:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5372
00000100:00080000:2.0:1775767995.402945:0:8:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767995.402948:0:8:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767995.402952:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775767995.402953:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114815104:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775767995.402965:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0700 x1862022114815232/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767995.402973:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767995.402976:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775767995.402978:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767995.402980:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5372
00000100:00080000:2.0:1775767995.402984:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767995.402988:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775767995.402993:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114815232:192.168.202.110@tcp:8:kworker.0
00000008:00100000:2.0:1775767995.402994:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775767995.403009:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0380 x1862022114815360/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767995.403020:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767995.403024:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775767995.403026:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775767995.403032:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5372
00000100:00100000:0.0:1775767995.403035:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114815360:192.168.202.110@tcp:38:kworker.0
00000100:00080000:2.0:1775767995.403038:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767995.403045:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767995.403051:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775767995.403062:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2a00 x1862022114815488/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775767995.403074:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775767995.403078:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775767995.403080:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775767995.403084:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114815488:192.168.202.110@tcp:38:kworker.0
00000100:00080000:2.0:1775767995.403088:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5372
00000100:00080000:2.0:1775767995.403095:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775767995.403099:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775767995.403105:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775767995.403118:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1500 x1862022114815616/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775767995.403161:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114815616:192.168.202.110@tcp:8:kworker.0
00010000:00010000:3.0:1775767995.658732:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767996.426665:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767996.682727:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767997.450713:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775767997.706696:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767998.474662:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000400:00000100:1.0:1775767998.474813:0:2391:0:(lib-socket.c:454:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.202.110:988
00000400:00000100:1.0:1775767998.474856:0:2391:0:(acceptor.c:111:lnet_connect_console_error()) Connection to 192.168.202.110@tcp at host 192.168.202.110 was unreachable: the network or that node may be down, or Lustre may be misconfigured.
00000800:00000100:1.0:1775767998.474874:0:2391:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:1.0:1775767998.474908:0:2391:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:1.0:1775767998.474926:0:2391:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:1.0:1775767998.474938:0:2391:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:1.0:1775767998.474948:0:2391:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:1.0:1775767998.474958:0:2391:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000100:00100000:0.0:1775767998.474963:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767995/real 1775767998]  req@ffff9f14852d0a80 x1862022114815104/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768020 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000800:00000100:1.0:1775767998.474971:0:2391:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:1.0:1775767998.474983:0:2391:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000100:00100000:0.0:1775767998.474997:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0a80 x1862022114815104/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768020 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767998.475011:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767998.475018:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767998.475022:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114815104:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775767998.475030:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767995/real 1775767998]  req@ffff9f14852d1f80 x1862022114814848/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768020 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767998.475045:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1f80 x1862022114814848/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768020 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767998.475052:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767998.475057:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767998.475060:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114814848:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767998.475068:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767995/real 1775767998]  req@ffff9f14852d3800 x1862022114814976/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768020 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767998.475081:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3800 x1862022114814976/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768020 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767998.475088:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767998.475092:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767998.475095:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114814976:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767998.475101:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767995/real 1775767998]  req@ffff9f14852d2680 x1862022114814720/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768020 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767998.475114:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2680 x1862022114814720/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768020 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767998.475121:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767998.475125:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767998.475128:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114814720:192.168.202.110@tcp:250:kworker.0
00000100:00100000:0.0:1775767998.475179:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767995/real 1775767998]  req@ffff9f14852d1500 x1862022114815616/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768020 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767998.475194:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1500 x1862022114815616/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768020 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767998.475202:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767998.475208:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767998.475212:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114815616:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775767998.475221:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767995/real 1775767998]  req@ffff9f14852d2a00 x1862022114815488/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768020 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767998.475235:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2a00 x1862022114815488/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768020 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767998.475243:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767998.475247:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767998.475250:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114815488:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767998.475258:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767995/real 1775767998]  req@ffff9f14852d0380 x1862022114815360/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768020 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767998.475271:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0380 x1862022114815360/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768020 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767998.475281:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767998.475285:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767998.475289:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114815360:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775767998.475296:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775767995/real 1775767998]  req@ffff9f14852d0700 x1862022114815232/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768020 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775767998.475307:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0700 x1862022114815232/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768020 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775767998.475314:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775767998.475318:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775767998.475322:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114815232:192.168.202.110@tcp:8:kworker.0
00010000:00010000:3.0:1775767998.730732:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775767999.498720:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775767999.754690:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1775768000.522669:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768000.522678:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1775768000.522682:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768000.522692:0:8:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5378
00000100:00080000:2.0:1775768000.522703:0:8:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768000.522709:0:8:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775768000.522733:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:2.0:1775768000.522748:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2d80 x1862022114815744/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768000.522768:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768000.522771:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775768000.522772:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768000.522776:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5378
00000100:00080000:2.0:1775768000.522780:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768000.522783:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768000.522796:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775768000.522806:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114815744:192.168.202.110@tcp:250:kworker.0
00000100:00100000:2.0:1775768000.522806:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3b80 x1862022114815872/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768000.522829:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768000.522831:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775768000.522832:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768000.522836:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5378
00000100:00080000:2.0:1775768000.522838:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768000.522841:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768000.522845:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775768000.522852:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0000 x1862022114816000/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768000.522860:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768000.522862:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775768000.522863:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768000.522866:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5378
00000100:00080000:2.0:1775768000.522868:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768000.522871:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768000.522874:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775768000.522880:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1180 x1862022114816128/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768000.522885:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768000.522887:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1775768000.522888:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768000.522891:0:8:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5378
00000100:00080000:2.0:1775768000.522894:0:8:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768000.522896:0:8:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768000.522899:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:0.0:1775768000.522912:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775768000.522914:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1880 x1862022114816256/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768000.522923:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768000.522927:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775768000.522928:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768000.522932:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5378
00000100:00080000:2.0:1775768000.522937:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768000.522939:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768000.522944:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775768000.522950:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0e00 x1862022114816384/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768000.522958:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768000.522959:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775768000.522961:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768000.522967:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5378
00000100:00080000:2.0:1775768000.522969:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768000.522972:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768000.522977:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775768000.522988:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768000/real 1775768000]  req@ffff9f14852d2d80 x1862022114815744/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768025 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775768000.522997:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2300 x1862022114816512/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768000.523019:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768000.523023:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775768000.523026:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775768000.523029:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2d80 x1862022114815744/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768025 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775768000.523035:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5378
00000100:00080000:2.0:1775768000.523048:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768000.523057:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00080000:0.0:1775768000.523059:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768000.523069:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768000.523075:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114815744:192.168.202.110@tcp:250:kworker.0
00000008:00100000:2.0:1775768000.523077:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775768000.523135:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3100 x1862022114816640/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775768000.523149:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114815872:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768000.523189:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768000.523209:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768000/real 1775768000]  req@ffff9f14852d3b80 x1862022114815872/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768025 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768000.523226:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3b80 x1862022114815872/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768025 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768000.523237:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768000.523241:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768000.523245:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114815872:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768000.523253:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114816000:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768000.523282:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768000.523295:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768000/real 1775768000]  req@ffff9f14852d0000 x1862022114816000/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768025 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768000.523309:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0000 x1862022114816000/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768025 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768000.523318:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768000.523323:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768000.523326:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114816000:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768000.523334:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114816128:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768000.523358:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768000.523369:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768000/real 1775768000]  req@ffff9f14852d1180 x1862022114816128/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768025 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768000.523382:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1180 x1862022114816128/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768025 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768000.523391:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768000.523395:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768000.523398:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114816128:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775768000.523405:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114816256:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768000.523430:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768000.523442:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768000/real 1775768000]  req@ffff9f14852d1880 x1862022114816256/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768025 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768000.523462:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1880 x1862022114816256/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768025 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768000.523471:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768000.523475:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768000.523478:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114816256:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775768000.523486:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114816384:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768000.523509:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768000.523522:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768000/real 1775768000]  req@ffff9f14852d0e00 x1862022114816384/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768025 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768000.523534:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0e00 x1862022114816384/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768025 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768000.523543:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768000.523546:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768000.523585:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114816384:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768000.523593:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114816512:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768000.523616:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768000.523628:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768000/real 1775768000]  req@ffff9f14852d2300 x1862022114816512/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768025 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768000.523641:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2300 x1862022114816512/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768025 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768000.523658:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768000.523662:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768000.523665:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114816512:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768000.523692:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114816640:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768000.523716:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768000.523729:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768000/real 1775768000]  req@ffff9f14852d3100 x1862022114816640/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768025 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768000.523741:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3100 x1862022114816640/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768025 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768000.523748:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768000.523752:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768000.523754:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114816640:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775768000.778767:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775768001.546802:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768001.802792:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768002.570670:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768002.826720:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768003.594657:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768003.850737:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768004.618658:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768004.874694:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1775768005.642689:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768005.642701:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1775768005.642706:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775768005.642713:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:2.0:1775768005.642718:0:8:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5383
00000100:00080000:2.0:1775768005.642731:0:8:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768005.642740:0:8:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775768005.642783:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3480 x1862022114816768/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768005.642809:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768005.642813:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775768005.642815:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768005.642819:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5383
00000100:00080000:2.0:1775768005.642823:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768005.642828:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768005.642838:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775768005.642845:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114816768:192.168.202.110@tcp:250:kworker.0
00000100:00100000:2.0:1775768005.642852:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1c00 x1862022114816896/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768005.642862:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768005.642867:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775768005.642869:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768005.642873:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5383
00000100:00080000:2.0:1775768005.642876:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768005.642880:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768005.642885:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775768005.642897:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3100 x1862022114817024/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768005.642906:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768005.642910:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775768005.642912:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768005.642915:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5383
00000100:00080000:2.0:1775768005.642919:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768005.642922:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000800:00000100:0.0:1775768005.642927:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000008:00100000:2.0:1775768005.642928:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775768005.642941:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2300 x1862022114817152/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768005.642950:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768005.642961:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1775768005.642962:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768005.642966:0:8:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5383
00000100:00080000:2.0:1775768005.642970:0:8:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768005.642973:0:8:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768005.642979:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775768005.642989:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768005/real 1775768005]  req@ffff9f14852d3480 x1862022114816768/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768030 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775768005.642992:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0e00 x1862022114817280/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768005.643016:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768005.643020:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775768005.643021:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775768005.643026:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3480 x1862022114816768/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768030 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775768005.643028:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5383
00000100:00080000:2.0:1775768005.643033:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768005.643041:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00080000:0.0:1775768005.643045:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000008:00100000:2.0:1775768005.643046:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00080000:0.0:1775768005.643055:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775768005.643059:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1880 x1862022114817408/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775768005.643064:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114816768:192.168.202.110@tcp:250:kworker.0
00000100:00080000:2.0:1775768005.643089:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768005.643093:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775768005.643095:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768005.643100:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5383
00000100:00080000:2.0:1775768005.643103:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768005.643107:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775768005.643110:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114816896:192.168.202.110@tcp:38:kworker.0
00000008:00100000:2.0:1775768005.643113:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775768005.643127:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1180 x1862022114817536/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768005.643137:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000800:00000100:0.0:1775768005.643142:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:2.0:1775768005.643143:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775768005.643147:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768005.643152:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5383
00000100:00080000:2.0:1775768005.643157:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1775768005.643160:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768005/real 1775768005]  req@ffff9f14852d1c00 x1862022114816896/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768030 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775768005.643163:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768005.643171:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775768005.643179:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1c00 x1862022114816896/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768030 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775768005.643184:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0000 x1862022114817664/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775768005.643201:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768005.643208:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768005.643212:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114816896:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768005.643220:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114817024:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768005.643239:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768005.643249:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768005/real 1775768005]  req@ffff9f14852d3100 x1862022114817024/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768030 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768005.643259:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3100 x1862022114817024/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768030 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768005.643267:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768005.643270:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768005.643274:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114817024:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768005.643279:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114817152:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768005.643297:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768005.643306:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768005/real 1775768005]  req@ffff9f14852d2300 x1862022114817152/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768030 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768005.643317:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2300 x1862022114817152/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768030 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768005.643324:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768005.643327:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768005.643330:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114817152:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775768005.643335:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114817280:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768005.643353:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768005.643363:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768005/real 1775768005]  req@ffff9f14852d0e00 x1862022114817280/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768030 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768005.643373:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0e00 x1862022114817280/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768030 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768005.643381:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768005.643385:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768005.643388:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114817280:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775768005.643393:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114817408:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768005.643409:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768005.643418:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768005/real 1775768005]  req@ffff9f14852d1880 x1862022114817408/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768030 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768005.643437:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1880 x1862022114817408/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768030 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768005.643445:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768005.643448:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768005.643451:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114817408:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768005.643469:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114817536:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768005.643487:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768005.643496:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768005/real 1775768005]  req@ffff9f14852d1180 x1862022114817536/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768030 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768005.643507:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1180 x1862022114817536/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768030 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768005.643515:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768005.643518:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768005.643521:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114817536:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768005.643526:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114817664:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768005.643542:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768005.643551:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768005/real 1775768005]  req@ffff9f14852d0000 x1862022114817664/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768030 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768005.643588:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0000 x1862022114817664/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768030 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768005.643602:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768005.643605:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768005.643607:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114817664:192.168.202.110@tcp:8:kworker.0
00010000:00010000:3.0:1775768005.898739:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775768006.666668:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775768006.922740:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775768007.690689:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775768007.946705:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775768008.714666:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768008.970702:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775768009.738725:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768009.994760:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775768010.762623:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:1.0:1775768010.762636:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775768010.762641:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1775768010.762644:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775768010.762651:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5388
00000100:00080000:1.0:1775768010.762660:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775768010.762665:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775768010.762698:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fca80 x1862022114817792/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775768010.762717:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775768010.762720:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775768010.762721:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775768010.762723:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5388
00000100:00080000:1.0:1775768010.762725:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775768010.762727:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775768010.762734:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775768010.762742:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ffb80 x1862022114817920/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775768010.762748:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775768010.762751:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775768010.762753:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775768010.762757:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114817792:192.168.202.110@tcp:250:kworker.0
00000100:00080000:1.0:1775768010.762757:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5388
00000100:00080000:1.0:1775768010.762759:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775768010.762764:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775768010.762767:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775768010.762773:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fed80 x1862022114818048/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775768010.762779:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775768010.762781:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775768010.762782:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775768010.762784:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5388
00000100:00080000:1.0:1775768010.762786:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775768010.762788:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775768010.762790:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775768010.762796:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff800 x1862022114818176/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775768010.762801:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775768010.762803:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1775768010.762804:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775768010.762806:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5388
00000100:00080000:1.0:1775768010.762808:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775768010.762809:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775768010.762812:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:0.0:1775768010.762817:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775768010.762822:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fc380 x1862022114818304/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775768010.762828:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775768010.762831:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775768010.762832:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775768010.762834:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5388
00000100:00080000:1.0:1775768010.762836:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775768010.762839:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775768010.762841:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775768010.762847:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff100 x1862022114818432/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775768010.762851:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775768010.762853:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775768010.762855:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775768010.762858:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768010/real 1775768010]  req@ffff9f14ab4fca80 x1862022114817792/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768035 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775768010.762859:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5388
00000100:00080000:1.0:1775768010.762866:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775768010.762870:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775768010.762874:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775768010.762884:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fca80 x1862022114817792/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768035 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768010.762892:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00100000:1.0:1775768010.762896:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fce00 x1862022114818560/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775768010.762898:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768010.762904:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114817792:192.168.202.110@tcp:250:kworker.0
00000100:00100000:0.0:1775768010.762914:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ffb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114817920:192.168.202.110@tcp:38:kworker.0
00000100:00080000:1.0:1775768010.762914:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775768010.762920:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775768010.762922:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775768010.762927:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5388
00000800:00000100:0.0:1775768010.762936:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:1.0:1775768010.762939:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775768010.762945:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775768010.762949:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768010/real 1775768010]  req@ffff9f14ab4ffb80 x1862022114817920/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768035 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:1.0:1775768010.762950:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775768010.762966:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdc00 x1862022114818688/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775768010.762970:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ffb80 x1862022114817920/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768035 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768010.762982:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768010.762986:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768010.762990:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ffb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114817920:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768010.763009:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114818048:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768010.763044:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768010.763051:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768010/real 1775768010]  req@ffff9f14ab4fed80 x1862022114818048/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768035 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768010.763060:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fed80 x1862022114818048/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768035 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768010.763066:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768010.763069:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768010.763072:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114818048:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768010.763078:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114818176:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768010.763097:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768010.763107:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768010/real 1775768010]  req@ffff9f14ab4ff800 x1862022114818176/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768035 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768010.763118:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff800 x1862022114818176/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768035 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768010.763125:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768010.763127:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768010.763129:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114818176:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775768010.763133:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114818304:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768010.763146:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768010.763152:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768010/real 1775768010]  req@ffff9f14ab4fc380 x1862022114818304/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768035 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768010.763167:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fc380 x1862022114818304/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768035 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768010.763175:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768010.763176:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768010.763178:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114818304:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775768010.763182:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114818432:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768010.763192:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768010.763198:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768010/real 1775768010]  req@ffff9f14ab4ff100 x1862022114818432/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768035 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768010.763204:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff100 x1862022114818432/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768035 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768010.763208:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768010.763210:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768010.763211:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114818432:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768010.763214:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114818560:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768010.763222:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768010.763227:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768010/real 1775768010]  req@ffff9f14ab4fce00 x1862022114818560/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768035 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768010.763233:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fce00 x1862022114818560/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768035 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768010.763237:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768010.763239:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768010.763240:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114818560:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768010.763243:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114818688:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768010.763253:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768010.763262:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768010/real 1775768010]  req@ffff9f14ab4fdc00 x1862022114818688/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768035 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768010.763272:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdc00 x1862022114818688/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768035 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768010.763280:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768010.763283:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768010.763285:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114818688:192.168.202.110@tcp:8:kworker.0
00010000:00010000:3.0:1775768011.018699:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775768011.786652:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768012.042695:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775768012.810628:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768013.066695:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768013.834674:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775768014.090712:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768014.858649:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775768015.114714:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1775768015.882669:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775768015.882678:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00100000:2.0:1775768015.882679:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:1.0:1775768015.882682:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775768015.882694:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5393
00000100:00080000:1.0:1775768015.882706:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775768015.882713:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775768015.882751:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fd880 x1862022114818816/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775768015.882774:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775768015.882777:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775768015.882779:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775768015.882782:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5393
00000100:00080000:1.0:1775768015.882784:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775768015.882786:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775768015.882794:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775768015.882805:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fe300 x1862022114818944/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775768015.882812:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775768015.882815:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775768015.882817:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775768015.882819:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5393
00000100:00080000:1.0:1775768015.882821:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775768015.882823:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775768015.882825:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775768015.882830:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114818816:192.168.202.110@tcp:250:kworker.0
00000100:00100000:1.0:1775768015.882834:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdf80 x1862022114819072/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775768015.882846:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775768015.882848:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775768015.882849:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775768015.882852:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5393
00000100:00080000:1.0:1775768015.882855:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775768015.882857:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775768015.882860:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775768015.882868:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fea00 x1862022114819200/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775768015.882874:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775768015.882876:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1775768015.882877:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775768015.882879:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5393
00000100:00080000:1.0:1775768015.882887:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775768015.882889:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775768015.882893:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:0.0:1775768015.882896:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775768015.882904:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdc00 x1862022114819328/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775768015.882911:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775768015.882914:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775768015.882915:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775768015.882918:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5393
00000100:00080000:1.0:1775768015.882919:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775768015.882922:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775768015.882924:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775768015.882931:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fce00 x1862022114819456/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775768015.882936:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775768015.882940:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00100000:0.0:1775768015.882941:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768015/real 1775768015]  req@ffff9f14ab4fd880 x1862022114818816/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768040 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775768015.882941:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775768015.882949:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5393
00000100:00080000:1.0:1775768015.882953:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775768015.882958:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775768015.882962:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775768015.882983:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff100 x1862022114819584/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775768015.882989:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775768015.882992:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00100000:0.0:1775768015.882995:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fd880 x1862022114818816/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768040 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775768015.882996:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775768015.883003:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5393
00000100:00080000:1.0:1775768015.883008:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775768015.883011:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775768015.883014:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00080000:0.0:1775768015.883017:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768015.883021:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114818816:192.168.202.110@tcp:250:kworker.0
00000008:00100000:1.0:1775768015.883022:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775768015.883032:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114818944:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775768015.883039:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fc380 x1862022114819712/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1775768015.883051:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768015.883060:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768015/real 1775768015]  req@ffff9f14ab4fe300 x1862022114818944/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768040 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768015.883068:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fe300 x1862022114818944/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768040 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768015.883074:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768015.883077:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768015.883079:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114818944:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768015.883097:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114819072:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768015.883107:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768015.883112:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768015/real 1775768015]  req@ffff9f14ab4fdf80 x1862022114819072/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768040 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768015.883119:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdf80 x1862022114819072/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768040 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768015.883127:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768015.883129:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768015.883131:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114819072:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768015.883134:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114819200:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768015.883142:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768015.883146:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768015/real 1775768015]  req@ffff9f14ab4fea00 x1862022114819200/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768040 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768015.883153:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fea00 x1862022114819200/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768040 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768015.883158:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768015.883159:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768015.883160:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114819200:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775768015.883163:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114819328:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768015.883171:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768015.883176:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768015/real 1775768015]  req@ffff9f14ab4fdc00 x1862022114819328/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768040 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768015.883183:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdc00 x1862022114819328/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768040 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768015.883187:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768015.883188:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768015.883191:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114819328:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775768015.883194:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114819456:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768015.883203:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768015.883207:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768015/real 1775768015]  req@ffff9f14ab4fce00 x1862022114819456/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768040 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768015.883213:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fce00 x1862022114819456/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768040 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768015.883217:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768015.883219:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768015.883220:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114819456:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768015.883223:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114819584:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768015.883231:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768015.883235:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768015/real 1775768015]  req@ffff9f14ab4ff100 x1862022114819584/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768040 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768015.883241:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff100 x1862022114819584/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768040 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768015.883245:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768015.883247:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768015.883249:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114819584:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768015.883252:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114819712:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768015.883260:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768015.883265:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768015/real 1775768015]  req@ffff9f14ab4fc380 x1862022114819712/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768040 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768015.883271:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fc380 x1862022114819712/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768040 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768015.883275:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768015.883277:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768015.883278:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114819712:192.168.202.110@tcp:8:kworker.0
00010000:00010000:3.0:1775768016.138783:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768016.906637:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768017.162710:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768017.930620:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768018.186726:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775768018.954666:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768019.210690:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775768019.978632:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768020.234676:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775768021.002675:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:0.0:1775768021.002742:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775768021.002750:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1775768021.002753:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775768021.002762:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5398
00000100:00080000:0.0:1775768021.002775:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775768021.002785:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775768021.002829:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3b80 x1862022114819840/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775768021.002862:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775768021.002868:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775768021.002870:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775768021.002877:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5398
00000100:00080000:0.0:1775768021.002882:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775768021.002888:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775768021.002898:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775768021.002912:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114819840:192.168.202.110@tcp:250:kworker.0
00000100:00100000:0.0:1775768021.002914:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d1f80 x1862022114819968/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775768021.002951:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775768021.002954:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:0.0:1775768021.002956:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775768021.002961:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5398
00000100:00080000:0.0:1775768021.002964:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775768021.002969:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775768021.002975:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775768021.002986:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0380 x1862022114820096/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775768021.002993:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775768021.003001:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000800:00000100:1.0:1775768021.003002:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:0.0:1775768021.003005:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775768021.003010:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5398
00000100:00080000:0.0:1775768021.003014:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775768021.003019:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775768021.003025:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775768021.003043:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768021/real 1775768021]  req@ffff9f14a04d3b80 x1862022114819840/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768046 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768021.003044:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3480 x1862022114820224/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775768021.003091:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775768021.003094:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1775768021.003096:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:1.0:1775768021.003098:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3b80 x1862022114819840/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768046 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768021.003101:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5398
00000100:00080000:0.0:1775768021.003109:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775768021.003114:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775768021.003124:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00080000:1.0:1775768021.003128:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00100000:0.0:1775768021.003143:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d1180 x1862022114820352/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775768021.003143:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775768021.003160:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114819840:192.168.202.110@tcp:250:kworker.0
00000100:00080000:0.0:1775768021.003163:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775768021.003169:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775768021.003171:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775768021.003174:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5398
00000100:00080000:0.0:1775768021.003176:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775768021.003178:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775768021.003181:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775768021.003192:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3100 x1862022114820480/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775768021.003200:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775768021.003203:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:0.0:1775768021.003205:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775768021.003209:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5398
00000100:00080000:0.0:1775768021.003211:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775768021.003218:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775768021.003219:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114819968:192.168.202.110@tcp:38:kworker.0
00000008:00100000:0.0:1775768021.003230:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775768021.003246:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0e00 x1862022114820608/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775768021.003256:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000800:00000100:1.0:1775768021.003262:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:0.0:1775768021.003263:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775768021.003265:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775768021.003274:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5398
00000100:00080000:0.0:1775768021.003280:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775768021.003286:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775768021.003287:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768021/real 1775768021]  req@ffff9f14a04d1f80 x1862022114819968/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768046 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:0.0:1775768021.003296:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775768021.003313:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d1500 x1862022114820736/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:1.0:1775768021.003322:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1f80 x1862022114819968/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768046 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775768021.003339:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775768021.003344:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775768021.003348:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114819968:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775768021.003368:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114820096:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775768021.003403:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775768021.003425:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768021/real 1775768021]  req@ffff9f14a04d0380 x1862022114820096/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768046 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775768021.003434:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0380 x1862022114820096/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768046 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775768021.003441:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775768021.003444:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775768021.003449:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114820096:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775768021.003455:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114820224:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775768021.003470:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775768021.003476:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768021/real 1775768021]  req@ffff9f14a04d3480 x1862022114820224/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768046 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775768021.003483:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3480 x1862022114820224/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768046 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775768021.003487:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775768021.003489:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775768021.003491:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114820224:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775768021.003495:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114820352:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775768021.003504:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775768021.003509:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768021/real 1775768021]  req@ffff9f14a04d1180 x1862022114820352/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768046 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775768021.003514:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1180 x1862022114820352/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768046 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775768021.003519:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775768021.003520:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775768021.003522:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114820352:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775768021.003525:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114820480:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775768021.003533:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775768021.003538:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768021/real 1775768021]  req@ffff9f14a04d3100 x1862022114820480/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768046 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775768021.003544:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3100 x1862022114820480/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768046 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775768021.003548:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775768021.003550:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775768021.003551:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114820480:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775768021.003595:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114820608:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775768021.003617:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775768021.003630:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768021/real 1775768021]  req@ffff9f14a04d0e00 x1862022114820608/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768046 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775768021.003642:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0e00 x1862022114820608/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768046 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775768021.003649:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775768021.003652:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775768021.003655:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114820608:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775768021.003660:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114820736:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775768021.003674:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775768021.003680:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768021/real 1775768021]  req@ffff9f14a04d1500 x1862022114820736/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768046 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775768021.003687:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1500 x1862022114820736/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768046 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775768021.003691:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775768021.003693:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775768021.003695:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114820736:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775768021.258760:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775768022.026636:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775768022.282728:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775768023.050687:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768023.306769:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775768024.074715:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768024.330810:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768025.098737:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768025.354802:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768026.122684:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:1.0:1775768026.122720:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775768026.122727:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1775768026.122731:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775768026.122741:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5403
00000100:00080000:1.0:1775768026.122751:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775768026.122756:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775768026.122793:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff800 x1862022114820864/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775768026.122819:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775768026.122821:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775768026.122823:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775768026.122826:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5403
00000100:00080000:1.0:1775768026.122828:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775768026.122831:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775768026.122840:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775768026.122849:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fed80 x1862022114820992/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775768026.122861:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1775768026.122863:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114820864:192.168.202.110@tcp:250:kworker.0
00000100:00080000:1.0:1775768026.122865:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775768026.122881:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775768026.122886:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5403
00000100:00080000:1.0:1775768026.122890:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775768026.122898:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775768026.122903:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775768026.122915:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ffb80 x1862022114821120/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775768026.122922:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775768026.122925:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775768026.122926:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775768026.122929:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5403
00000100:00080000:1.0:1775768026.122931:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775768026.122934:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775768026.122937:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775768026.122944:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fca80 x1862022114821248/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775768026.122950:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775768026.122952:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1775768026.122953:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775768026.122955:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5403
00000100:00080000:1.0:1775768026.122964:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775768026.122967:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775768026.122970:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:0.0:1775768026.122976:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775768026.122982:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fc380 x1862022114821376/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775768026.122993:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775768026.122997:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775768026.122999:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775768026.123004:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5403
00000100:00080000:1.0:1775768026.123009:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775768026.123014:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775768026.123021:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775768026.123032:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768026/real 1775768026]  req@ffff9f14ab4ff800 x1862022114820864/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768051 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775768026.123038:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff100 x1862022114821504/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775768026.123087:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775768026.123090:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775768026.123091:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775768026.123099:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5403
00000100:00100000:0.0:1775768026.123100:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff800 x1862022114820864/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768051 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775768026.123108:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775768026.123120:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00080000:0.0:1775768026.123128:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000008:00100000:1.0:1775768026.123129:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00080000:0.0:1775768026.123137:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768026.123143:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114820864:192.168.202.110@tcp:250:kworker.0
00000100:00100000:1.0:1775768026.123149:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fce00 x1862022114821632/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775768026.123169:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775768026.123172:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775768026.123174:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775768026.123177:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5403
00000100:00080000:1.0:1775768026.123179:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775768026.123181:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775768026.123187:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775768026.123191:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114820992:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775768026.123198:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdc00 x1862022114821760/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1775768026.123221:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768026.123240:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768026/real 1775768026]  req@ffff9f14ab4fed80 x1862022114820992/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768051 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768026.123250:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fed80 x1862022114820992/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768051 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768026.123256:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768026.123259:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768026.123261:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114820992:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768026.123265:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ffb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114821120:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768026.123278:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768026.123283:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768026/real 1775768026]  req@ffff9f14ab4ffb80 x1862022114821120/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768051 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768026.123291:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ffb80 x1862022114821120/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768051 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768026.123296:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768026.123298:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768026.123300:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ffb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114821120:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768026.123304:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114821248:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768026.123314:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768026.123330:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768026/real 1775768026]  req@ffff9f14ab4fca80 x1862022114821248/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768051 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768026.123339:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fca80 x1862022114821248/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768051 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768026.123345:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768026.123347:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768026.123349:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114821248:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775768026.123353:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114821376:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768026.123364:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768026.123369:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768026/real 1775768026]  req@ffff9f14ab4fc380 x1862022114821376/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768051 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768026.123377:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fc380 x1862022114821376/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768051 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768026.123382:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768026.123384:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768026.123386:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114821376:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775768026.123389:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114821504:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768026.123399:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768026.123405:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768026/real 1775768026]  req@ffff9f14ab4ff100 x1862022114821504/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768051 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768026.123412:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff100 x1862022114821504/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768051 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768026.123423:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768026.123425:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768026.123427:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114821504:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768026.123430:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114821632:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768026.123440:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768026.123445:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768026/real 1775768026]  req@ffff9f14ab4fce00 x1862022114821632/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768051 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768026.123452:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fce00 x1862022114821632/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768051 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768026.123456:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768026.123458:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768026.123459:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114821632:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768026.123473:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114821760:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768026.123483:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768026.123489:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768026/real 1775768026]  req@ffff9f14ab4fdc00 x1862022114821760/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768051 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768026.123496:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdc00 x1862022114821760/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768051 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768026.123500:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768026.123502:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768026.123505:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114821760:192.168.202.110@tcp:8:kworker.0
00010000:00010000:3.0:1775768026.378718:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768027.146689:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768027.402730:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768028.170638:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775768028.426728:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775768029.194698:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775768029.450721:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775768030.218691:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768030.474801:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1775768031.242714:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775768031.242725:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00100000:0.0:1775768031.242728:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775768031.242730:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768031.242747:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5408
00000100:00080000:3.0:1775768031.242769:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768031.242784:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775768031.242839:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745500 x1862022114821888/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775768031.242870:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775768031.242874:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775768031.242875:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768031.242880:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5408
00000100:00080000:3.0:1775768031.242887:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768031.242892:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775768031.242902:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775768031.242910:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114821888:192.168.202.110@tcp:250:kworker.0
00000100:00100000:3.0:1775768031.242917:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744e00 x1862022114822016/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775768031.242926:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775768031.242929:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775768031.242930:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768031.242935:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5408
00000100:00080000:3.0:1775768031.242938:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768031.242940:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775768031.242943:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775768031.242951:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744380 x1862022114822144/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775768031.242957:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775768031.242959:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775768031.242961:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768031.242964:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5408
00000100:00080000:3.0:1775768031.242966:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768031.242969:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775768031.242972:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775768031.242978:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746a00 x1862022114822272/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775768031.242985:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775768031.242987:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775768031.242988:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768031.242991:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5408
00000100:00080000:3.0:1775768031.242995:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000800:00000100:0.0:1775768031.243002:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775768031.243008:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775768031.243014:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775768031.243022:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744a80 x1862022114822400/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775768031.243029:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775768031.243032:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775768031.243034:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768031.243037:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5408
00000100:00080000:3.0:1775768031.243040:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768031.243043:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775768031.243046:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775768031.243058:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747480 x1862022114822528/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775768031.243069:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768031/real 1775768031]  req@ffff9f1485745500 x1862022114821888/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768056 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775768031.243075:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775768031.243086:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775768031.243089:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768031.243095:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5408
00000100:00080000:3.0:1775768031.243102:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768031.243107:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775768031.243115:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775768031.243131:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745880 x1862022114822656/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775768031.243132:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745500 x1862022114821888/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768056 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775768031.243143:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775768031.243145:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775768031.243148:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768031.243158:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5408
00000100:00080000:3.0:1775768031.243163:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768031.243169:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775768031.243176:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00080000:0.0:1775768031.243181:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00100000:3.0:1775768031.243183:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746300 x1862022114822784/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775768031.243194:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768031.243204:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114821888:192.168.202.110@tcp:250:kworker.0
00000100:00100000:0.0:1775768031.243233:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114822016:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768031.243266:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768031.243277:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768031/real 1775768031]  req@ffff9f1485744e00 x1862022114822016/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768056 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768031.243286:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744e00 x1862022114822016/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768056 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768031.243293:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768031.243296:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768031.243298:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114822016:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768031.243305:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114822144:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768031.243318:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768031.243324:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768031/real 1775768031]  req@ffff9f1485744380 x1862022114822144/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768056 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768031.243339:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744380 x1862022114822144/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768056 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768031.243344:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768031.243346:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768031.243349:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114822144:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768031.243353:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114822272:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768031.243365:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768031.243370:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768031/real 1775768031]  req@ffff9f1485746a00 x1862022114822272/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768056 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768031.243378:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746a00 x1862022114822272/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768056 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768031.243384:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768031.243386:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768031.243388:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114822272:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775768031.243392:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114822400:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768031.243401:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768031.243407:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768031/real 1775768031]  req@ffff9f1485744a80 x1862022114822400/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768056 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768031.243415:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744a80 x1862022114822400/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768056 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768031.243420:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768031.243422:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768031.243424:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114822400:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775768031.243427:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114822528:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768031.243438:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768031.243443:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768031/real 1775768031]  req@ffff9f1485747480 x1862022114822528/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768056 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768031.243449:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747480 x1862022114822528/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768056 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768031.243454:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768031.243457:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768031.243458:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114822528:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768031.243462:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114822656:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768031.243473:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768031.243479:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768031/real 1775768031]  req@ffff9f1485745880 x1862022114822656/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768056 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768031.243486:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745880 x1862022114822656/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768056 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768031.243496:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768031.243499:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768031.243501:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114822656:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768031.243504:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114822784:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768031.243514:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768031.243519:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768031/real 1775768031]  req@ffff9f1485746300 x1862022114822784/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768056 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768031.243526:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746300 x1862022114822784/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768056 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768031.243530:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768031.243533:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768031.243535:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114822784:192.168.202.110@tcp:8:kworker.0
00010000:00010000:3.0:1775768031.498784:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775768032.266697:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768032.522713:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775768033.290633:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768033.546710:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775768034.314633:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768034.570723:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775768035.338717:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775768035.594742:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1775768036.362695:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:2.0:1775768036.362712:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775768036.362712:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775768036.362717:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768036.362730:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5413
00000100:00080000:3.0:1775768036.362744:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768036.362753:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775768036.362808:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747b80 x1862022114822912/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775768036.362837:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775768036.362841:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775768036.362843:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768036.362848:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5413
00000100:00080000:3.0:1775768036.362852:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768036.362857:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775768036.362868:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775768036.362871:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114822912:192.168.202.110@tcp:250:kworker.0
00000100:00100000:3.0:1775768036.362886:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746300 x1862022114823040/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775768036.362898:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775768036.362902:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775768036.362904:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768036.362909:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5413
00000100:00080000:3.0:1775768036.362913:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768036.362917:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775768036.362921:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775768036.362931:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745880 x1862022114823168/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775768036.362942:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775768036.362946:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000800:00000100:0.0:1775768036.362950:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775768036.362951:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768036.362958:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5413
00000100:00080000:3.0:1775768036.362962:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768036.362966:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775768036.362972:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775768036.362983:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747480 x1862022114823296/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775768036.362994:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775768036.362999:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00100000:0.0:1775768036.363000:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768036/real 1775768036]  req@ffff9f1485747b80 x1862022114822912/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768061 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775768036.363004:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768036.363013:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5413
00000100:00080000:3.0:1775768036.363018:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768036.363023:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775768036.363026:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747b80 x1862022114822912/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768061 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:3.0:1775768036.363031:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00080000:0.0:1775768036.363044:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00100000:3.0:1775768036.363047:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744a80 x1862022114823424/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775768036.363051:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768036.363059:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114822912:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775768036.363070:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775768036.363083:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00100000:0.0:1775768036.363084:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114823040:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775768036.363086:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768036.363093:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5413
00000100:00080000:3.0:1775768036.363099:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000800:00000100:0.0:1775768036.363106:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775768036.363107:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775768036.363115:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775768036.363120:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768036/real 1775768036]  req@ffff9f1485746300 x1862022114823040/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768061 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775768036.363129:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746a00 x1862022114823552/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775768036.363134:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746300 x1862022114823040/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768061 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775768036.363153:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775768036.363155:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775768036.363158:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775768036.363161:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775768036.363163:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00080000:3.0:1775768036.363169:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5413
00000100:00100000:0.0:1775768036.363172:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114823040:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775768036.363175:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768036.363182:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775768036.363187:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114823168:192.168.202.110@tcp:38:kworker.0
00000008:00100000:3.0:1775768036.363188:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775768036.363205:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744380 x1862022114823680/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775768036.363215:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000800:00000100:0.0:1775768036.363221:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775768036.363222:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775768036.363225:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768036.363231:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5413
00000100:00080000:3.0:1775768036.363235:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1775768036.363236:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768036/real 1775768036]  req@ffff9f1485745880 x1862022114823168/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768061 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775768036.363243:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775768036.363252:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775768036.363254:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745880 x1862022114823168/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768061 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768036.363262:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00100000:3.0:1775768036.363267:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744e00 x1862022114823808/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775768036.363268:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768036.363275:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114823168:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768036.363287:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114823296:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768036.363310:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768036.363317:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768036/real 1775768036]  req@ffff9f1485747480 x1862022114823296/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768061 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768036.363324:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747480 x1862022114823296/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768061 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768036.363330:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768036.363333:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768036.363335:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114823296:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775768036.363338:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114823424:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768036.363348:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768036.363353:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768036/real 1775768036]  req@ffff9f1485744a80 x1862022114823424/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768061 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768036.363359:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744a80 x1862022114823424/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768061 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768036.363365:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768036.363366:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768036.363368:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114823424:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775768036.363380:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114823552:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768036.363390:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768036.363396:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768036/real 1775768036]  req@ffff9f1485746a00 x1862022114823552/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768061 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768036.363404:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746a00 x1862022114823552/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768061 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768036.363409:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768036.363411:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768036.363413:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114823552:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768036.363416:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114823680:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768036.363426:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768036.363432:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768036/real 1775768036]  req@ffff9f1485744380 x1862022114823680/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768061 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768036.363439:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744380 x1862022114823680/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768061 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768036.363444:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768036.363445:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768036.363447:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114823680:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768036.363450:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114823808:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768036.363459:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768036.363464:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768036/real 1775768036]  req@ffff9f1485744e00 x1862022114823808/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768061 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768036.363471:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744e00 x1862022114823808/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768061 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768036.363481:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768036.363483:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768036.363485:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114823808:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775768036.618718:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768037.386683:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768037.642738:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768038.410683:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768038.666745:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775768039.434959:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768039.690780:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775768040.458726:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768040.714772:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1775768041.482688:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1775768041.482691:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:1.0:1775768041.482700:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1775768041.482705:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775768041.482714:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5418
00000100:00080000:1.0:1775768041.482741:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775768041.482749:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775768041.482793:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fea00 x1862022114823936/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775768041.482818:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775768041.482821:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775768041.482822:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775768041.482825:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5418
00000100:00080000:1.0:1775768041.482828:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775768041.482831:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775768041.482840:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775768041.482851:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114823936:192.168.202.110@tcp:250:kworker.0
00000100:00100000:1.0:1775768041.482855:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdf80 x1862022114824064/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775768041.482865:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775768041.482868:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775768041.482869:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775768041.482872:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5418
00000100:00080000:1.0:1775768041.482874:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775768041.482876:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775768041.482879:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775768041.482887:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fe300 x1862022114824192/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775768041.482893:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775768041.482896:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775768041.482898:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775768041.482901:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5418
00000100:00080000:1.0:1775768041.482916:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775768041.482919:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775768041.482922:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:0.0:1775768041.482928:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775768041.482932:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fd880 x1862022114824320/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775768041.482939:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775768041.482942:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1775768041.482944:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775768041.482946:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5418
00000100:00080000:1.0:1775768041.482950:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775768041.482953:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775768041.482956:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775768041.482963:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdc00 x1862022114824448/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775768041.482970:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775768041.482972:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775768041.482973:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775768041.482977:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768041/real 1775768041]  req@ffff9f14ab4fea00 x1862022114823936/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768066 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775768041.482980:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5418
00000100:00080000:1.0:1775768041.482985:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775768041.482992:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775768041.482997:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775768041.483005:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fea00 x1862022114823936/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768066 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775768041.483008:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fce00 x1862022114824576/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775768041.483032:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775768041.483033:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775768041.483038:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:0.0:1775768041.483042:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:1.0:1775768041.483042:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775768041.483048:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114823936:192.168.202.110@tcp:250:kworker.0
00000100:00080000:1.0:1775768041.483051:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5418
00000100:00080000:1.0:1775768041.483057:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775768041.483061:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775768041.483066:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775768041.483072:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114824064:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775768041.483082:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff100 x1862022114824704/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1775768041.483094:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:1.0:1775768041.483094:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775768041.483100:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775768041.483102:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775768041.483107:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768041/real 1775768041]  req@ffff9f14ab4fdf80 x1862022114824064/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768066 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775768041.483109:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5418
00000100:00080000:1.0:1775768041.483117:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1775768041.483122:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdf80 x1862022114824064/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768066 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775768041.483124:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775768041.483133:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00080000:0.0:1775768041.483143:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768041.483148:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775768041.483148:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fc380 x1862022114824832/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775768041.483161:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114824064:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768041.483168:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114824192:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768041.483196:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768041.483208:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768041/real 1775768041]  req@ffff9f14ab4fe300 x1862022114824192/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768066 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768041.483221:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fe300 x1862022114824192/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768066 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768041.483231:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768041.483234:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768041.483238:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114824192:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768041.483244:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114824320:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768041.483266:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768041.483277:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768041/real 1775768041]  req@ffff9f14ab4fd880 x1862022114824320/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768066 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768041.483290:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fd880 x1862022114824320/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768066 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768041.483298:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768041.483301:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768041.483305:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114824320:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775768041.483312:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114824448:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768041.483332:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768041.483344:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768041/real 1775768041]  req@ffff9f14ab4fdc00 x1862022114824448/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768066 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768041.483356:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdc00 x1862022114824448/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768066 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768041.483365:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768041.483369:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768041.483372:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114824448:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775768041.483379:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114824576:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768041.483399:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768041.483410:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768041/real 1775768041]  req@ffff9f14ab4fce00 x1862022114824576/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768066 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768041.483423:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fce00 x1862022114824576/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768066 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768041.483432:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768041.483436:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768041.483439:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114824576:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768041.483459:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114824704:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768041.483481:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768041.483493:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768041/real 1775768041]  req@ffff9f14ab4ff100 x1862022114824704/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768066 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768041.483507:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff100 x1862022114824704/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768066 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768041.483516:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768041.483519:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768041.483522:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114824704:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768041.483528:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114824832:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768041.483547:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768041.483584:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768041/real 1775768041]  req@ffff9f14ab4fc380 x1862022114824832/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768066 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768041.483598:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fc380 x1862022114824832/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768066 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768041.483606:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768041.483610:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768041.483613:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114824832:192.168.202.110@tcp:8:kworker.0
00010000:00010000:3.0:1775768041.738792:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775768042.506767:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775768042.762807:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775768043.530664:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775768043.786735:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775768044.554687:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768044.810760:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775768045.578739:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768045.834711:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775768046.602720:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:2.0:1775768046.602720:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768046.602741:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1775768046.602746:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768046.602758:0:8:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5424
00000100:00080000:2.0:1775768046.602772:0:8:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768046.602780:0:8:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775768046.602825:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3b80 x1862022114824960/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768046.602858:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768046.602862:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775768046.602864:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768046.602868:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5424
00000100:00080000:2.0:1775768046.602872:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768046.602876:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768046.602887:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775768046.602902:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114824960:192.168.202.110@tcp:250:kworker.0
00000100:00100000:2.0:1775768046.602903:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2d80 x1862022114825088/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768046.602919:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768046.602921:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775768046.602922:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768046.602926:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5424
00000100:00080000:2.0:1775768046.602930:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768046.602939:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768046.602946:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775768046.602960:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0700 x1862022114825216/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768046.602970:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768046.602974:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775768046.602976:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768046.602980:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5424
00000100:00080000:2.0:1775768046.602983:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768046.602990:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000800:00000100:0.0:1775768046.602991:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000008:00100000:2.0:1775768046.602997:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775768046.603012:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0380 x1862022114825344/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768046.603022:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768046.603026:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1775768046.603028:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768046.603032:0:8:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5424
00000100:00080000:2.0:1775768046.603035:0:8:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768046.603038:0:8:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768046.603042:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775768046.603052:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2a00 x1862022114825472/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768046.603058:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1775768046.603060:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768046/real 1775768046]  req@ffff9f14852d3b80 x1862022114824960/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768071 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775768046.603061:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775768046.603064:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768046.603071:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5424
00000100:00080000:2.0:1775768046.603075:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768046.603081:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768046.603084:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775768046.603093:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3b80 x1862022114824960/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768071 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775768046.603093:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1500 x1862022114825600/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768046.603111:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775768046.603117:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775768046.603122:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775768046.603124:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768046.603131:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5424
00000100:00080000:0.0:1775768046.603132:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:2.0:1775768046.603138:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1775768046.603139:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114824960:192.168.202.110@tcp:250:kworker.0
00000100:00080000:2.0:1775768046.603146:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768046.603151:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775768046.603164:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2680 x1862022114825728/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775768046.603168:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114825088:192.168.202.110@tcp:38:kworker.0
00000100:00080000:2.0:1775768046.603189:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768046.603192:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775768046.603193:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768046.603197:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5424
00000100:00080000:2.0:1775768046.603200:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768046.603204:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000800:00000100:0.0:1775768046.603209:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000008:00100000:2.0:1775768046.603211:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775768046.603224:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768046/real 1775768046]  req@ffff9f14852d2d80 x1862022114825088/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768071 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775768046.603224:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3800 x1862022114825856/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775768046.603254:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2d80 x1862022114825088/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768071 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768046.603263:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768046.603267:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768046.603270:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114825088:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768046.603276:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114825216:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768046.603292:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768046.603301:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768046/real 1775768046]  req@ffff9f14852d0700 x1862022114825216/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768071 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768046.603312:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0700 x1862022114825216/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768071 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768046.603325:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768046.603328:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768046.603331:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114825216:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768046.603337:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114825344:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768046.603359:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768046.603368:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768046/real 1775768046]  req@ffff9f14852d0380 x1862022114825344/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768071 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768046.603379:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0380 x1862022114825344/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768071 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768046.603387:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768046.603390:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768046.603393:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114825344:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775768046.603399:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114825472:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768046.603414:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768046.603421:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768046/real 1775768046]  req@ffff9f14852d2a00 x1862022114825472/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768071 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768046.603432:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2a00 x1862022114825472/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768071 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768046.603439:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768046.603442:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768046.603446:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114825472:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775768046.603452:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114825600:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768046.603467:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768046.603475:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768046/real 1775768046]  req@ffff9f14852d1500 x1862022114825600/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768071 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768046.603485:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1500 x1862022114825600/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768071 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768046.603493:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768046.603496:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768046.603499:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114825600:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768046.603516:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114825728:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768046.603530:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768046.603538:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768046/real 1775768046]  req@ffff9f14852d2680 x1862022114825728/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768071 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768046.603549:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2680 x1862022114825728/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768071 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768046.603582:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768046.603586:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768046.603589:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114825728:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768046.603594:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114825856:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768046.603610:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768046.603620:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768046/real 1775768046]  req@ffff9f14852d3800 x1862022114825856/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768071 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768046.603638:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3800 x1862022114825856/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768071 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768046.603647:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768046.603649:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768046.603652:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114825856:192.168.202.110@tcp:8:kworker.0
00010000:00010000:3.0:1775768046.858825:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775768047.626637:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768047.882772:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775768048.650686:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768048.906756:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775768049.674713:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775768049.930847:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775768050.698665:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775768050.954693:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775768051.722654:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775768051.722663:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775768051.722667:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775768051.722671:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768051.722678:0:8:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5429
00000100:00080000:3.0:1775768051.722687:0:8:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768051.722692:0:8:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775768051.722727:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745500 x1862022114825984/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775768051.722752:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775768051.722754:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775768051.722756:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768051.722758:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5429
00000100:00080000:3.0:1775768051.722761:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768051.722764:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775768051.722772:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775768051.722785:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746680 x1862022114826112/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775768051.722795:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775768051.722799:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775768051.722802:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775768051.722803:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114825984:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775768051.722810:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5429
00000100:00080000:3.0:1775768051.722814:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768051.722818:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775768051.722823:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775768051.722833:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745c00 x1862022114826240/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775768051.722839:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775768051.722842:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775768051.722843:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768051.722847:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5429
00000100:00080000:3.0:1775768051.722850:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768051.722854:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775768051.722859:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:0.0:1775768051.722868:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775768051.722872:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745180 x1862022114826368/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775768051.722882:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775768051.722885:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775768051.722887:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768051.722891:0:8:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5429
00000100:00080000:3.0:1775768051.722895:0:8:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768051.722898:0:8:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775768051.722903:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775768051.722907:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768051/real 1775768051]  req@ffff9f1485745500 x1862022114825984/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768076 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775768051.722916:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747100 x1862022114826496/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775768051.722923:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1775768051.722925:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745500 x1862022114825984/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768076 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775768051.722926:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775768051.722929:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775768051.722939:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768051.722943:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:3.0:1775768051.722946:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5429
00000100:00100000:0.0:1775768051.722951:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114825984:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775768051.722952:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768051.722961:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775768051.722966:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775768051.722978:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744000 x1862022114826624/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775768051.722980:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114826112:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775768051.722997:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775768051.723000:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775768051.723002:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768051.723005:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5429
00000100:00080000:3.0:1775768051.723008:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768051.723013:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000800:00000100:0.0:1775768051.723015:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000008:00100000:3.0:1775768051.723018:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775768051.723028:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747800 x1862022114826752/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775768051.723034:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768051/real 1775768051]  req@ffff9f1485746680 x1862022114826112/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768076 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775768051.723039:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775768051.723044:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775768051.723047:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768051.723054:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5429
00000100:00080000:3.0:1775768051.723057:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768051.723060:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775768051.723061:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746680 x1862022114826112/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768076 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:3.0:1775768051.723063:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775768051.723070:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745f80 x1862022114826880/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775768051.723096:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768051.723101:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768051.723104:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114826112:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768051.723109:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114826240:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768051.723125:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768051.723132:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768051/real 1775768051]  req@ffff9f1485745c00 x1862022114826240/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768076 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768051.723140:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745c00 x1862022114826240/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768076 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768051.723144:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768051.723146:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768051.723148:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114826240:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768051.723151:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114826368:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768051.723160:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768051.723165:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768051/real 1775768051]  req@ffff9f1485745180 x1862022114826368/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768076 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768051.723171:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745180 x1862022114826368/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768076 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768051.723176:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768051.723178:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768051.723180:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114826368:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775768051.723183:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114826496:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768051.723191:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768051.723198:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768051/real 1775768051]  req@ffff9f1485747100 x1862022114826496/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768076 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768051.723207:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747100 x1862022114826496/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768076 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768051.723214:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768051.723217:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768051.723220:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114826496:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775768051.723235:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114826624:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768051.723255:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768051.723265:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768051/real 1775768051]  req@ffff9f1485744000 x1862022114826624/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768076 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768051.723276:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744000 x1862022114826624/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768076 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768051.723283:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768051.723286:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768051.723289:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114826624:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768051.723295:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114826752:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768051.723312:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768051.723321:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768051/real 1775768051]  req@ffff9f1485747800 x1862022114826752/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768076 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768051.723331:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747800 x1862022114826752/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768076 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768051.723339:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768051.723342:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768051.723346:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114826752:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768051.723351:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114826880:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768051.723368:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768051.723378:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768051/real 1775768051]  req@ffff9f1485745f80 x1862022114826880/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768076 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768051.723387:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745f80 x1862022114826880/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768076 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768051.723392:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768051.723393:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768051.723395:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114826880:192.168.202.110@tcp:8:kworker.0
00010000:00010000:3.0:1775768051.978708:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775768052.746690:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768053.002725:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775768053.770681:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768054.026722:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775768054.794652:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768055.050706:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775768055.818694:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768056.074717:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775768056.842654:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:2.0:1775768056.842685:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768056.842690:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1775768056.842693:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768056.842701:0:8:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5434
00000100:00080000:2.0:1775768056.842710:0:8:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768056.842715:0:8:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775768056.842748:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1f80 x1862022114827008/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768056.842776:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768056.842779:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775768056.842781:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768056.842785:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5434
00000100:00080000:2.0:1775768056.842790:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768056.842795:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775768056.842805:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114827008:192.168.202.110@tcp:250:kworker.0
00000008:00100000:2.0:1775768056.842806:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775768056.842824:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0a80 x1862022114827136/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768056.842836:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768056.842839:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775768056.842841:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768056.842845:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5434
00000100:00080000:2.0:1775768056.842849:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768056.842852:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768056.842856:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000800:00000100:3.0:1775768056.842861:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775768056.842869:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3800 x1862022114827264/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768056.842876:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768056.842878:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775768056.842879:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768056.842881:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5434
00000100:00080000:2.0:1775768056.842883:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768056.842886:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768056.842889:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775768056.842896:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2680 x1862022114827392/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768056.842901:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768056.842903:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1775768056.842904:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768056.842906:0:8:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5434
00000100:00080000:2.0:1775768056.842908:0:8:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768056.842909:0:8:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768056.842912:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775768056.842916:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1500 x1862022114827520/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768056.842923:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768056.842926:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775768056.842928:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768056.842931:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5434
00000100:00080000:2.0:1775768056.842934:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768056.842939:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768056.842943:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775768056.842947:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768056/real 1775768056]  req@ffff9f14852d1f80 x1862022114827008/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768081 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775768056.842957:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2a00 x1862022114827648/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768056.842967:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768056.842971:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775768056.842973:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768056.842977:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5434
00000100:00080000:2.0:1775768056.842981:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768056.842984:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768056.842989:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775768056.843001:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0380 x1862022114827776/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768056.843009:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768056.843012:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775768056.843014:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768056.843017:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5434
00000100:00080000:2.0:1775768056.843021:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768056.843025:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768056.843030:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775768056.843042:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0700 x1862022114827904/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775768056.843054:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1f80 x1862022114827008/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768081 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775768056.843063:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775768056.843068:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775768056.843072:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114827008:192.168.202.110@tcp:250:kworker.0
00000100:00100000:3.0:1775768056.843132:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114827136:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775768056.843150:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775768056.843158:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768056/real 1775768056]  req@ffff9f14852d0a80 x1862022114827136/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768081 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775768056.843165:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0a80 x1862022114827136/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768081 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775768056.843170:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775768056.843172:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775768056.843174:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114827136:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775768056.843177:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114827264:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775768056.843186:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775768056.843191:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768056/real 1775768056]  req@ffff9f14852d3800 x1862022114827264/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768081 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775768056.843197:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3800 x1862022114827264/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768081 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775768056.843201:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775768056.843203:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775768056.843204:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114827264:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775768056.843208:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114827392:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775768056.843216:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775768056.843221:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768056/real 1775768056]  req@ffff9f14852d2680 x1862022114827392/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768081 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775768056.843228:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2680 x1862022114827392/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768081 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775768056.843232:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775768056.843234:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775768056.843235:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114827392:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775768056.843238:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114827520:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775768056.843246:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775768056.843250:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768056/real 1775768056]  req@ffff9f14852d1500 x1862022114827520/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768081 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775768056.843256:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1500 x1862022114827520/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768081 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775768056.843261:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775768056.843262:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775768056.843264:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114827520:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775768056.843266:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114827648:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775768056.843274:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775768056.843279:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768056/real 1775768056]  req@ffff9f14852d2a00 x1862022114827648/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768081 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775768056.843285:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2a00 x1862022114827648/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768081 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775768056.843293:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775768056.843295:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775768056.843296:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114827648:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775768056.843299:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114827776:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775768056.843307:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775768056.843311:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768056/real 1775768056]  req@ffff9f14852d0380 x1862022114827776/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768081 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775768056.843317:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0380 x1862022114827776/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768081 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775768056.843320:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775768056.843322:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775768056.843324:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114827776:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775768056.843327:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114827904:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775768056.843335:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775768056.843339:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768056/real 1775768056]  req@ffff9f14852d0700 x1862022114827904/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768081 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775768056.843344:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0700 x1862022114827904/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768081 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775768056.843349:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775768056.843350:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775768056.843352:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114827904:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775768057.098737:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775768057.866686:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775768058.122686:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775768058.890672:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768059.146706:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775768059.914661:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768060.170678:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775768060.938613:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768061.194712:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1775768061.962625:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775768061.962632:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1775768061.962636:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775768061.962645:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5439
00000100:00100000:2.0:1775768061.962645:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:0.0:1775768061.962654:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775768061.962660:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775768061.962697:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0000 x1862022114828032/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775768061.962725:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775768061.962728:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775768061.962729:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775768061.962732:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5439
00000100:00080000:0.0:1775768061.962735:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775768061.962737:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775768061.962746:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775768061.962759:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0a80 x1862022114828160/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775768061.962770:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114828032:192.168.202.110@tcp:250:kworker.0
00000100:00080000:0.0:1775768061.962772:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775768061.962779:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:0.0:1775768061.962782:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775768061.962787:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5439
00000100:00080000:0.0:1775768061.962790:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775768061.962794:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775768061.962799:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775768061.962810:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d2300 x1862022114828288/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775768061.962817:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775768061.962819:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775768061.962821:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775768061.962823:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5439
00000100:00080000:0.0:1775768061.962826:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775768061.962828:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775768061.962832:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775768061.962838:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d2a00 x1862022114828416/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775768061.962844:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775768061.962846:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1775768061.962847:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775768061.962849:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5439
00000100:00080000:0.0:1775768061.962851:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775768061.962853:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775768061.962856:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775768061.962863:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3800 x1862022114828544/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775768061.962870:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:3.0:1775768061.962871:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114828160:192.168.202.110@tcp:38:kworker.0
00000100:00080000:0.0:1775768061.962874:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775768061.962877:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775768061.962881:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5439
00000100:00080000:0.0:1775768061.962883:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775768061.962886:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775768061.962889:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775768061.962898:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d1500 x1862022114828672/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775768061.962903:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:3.0:1775768061.962904:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114828288:192.168.202.110@tcp:38:kworker.0
00000100:00080000:0.0:1775768061.962908:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:0.0:1775768061.962911:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775768061.962916:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5439
00000100:00080000:0.0:1775768061.962918:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775768061.962921:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775768061.962924:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775768061.962933:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114828416:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775768061.962934:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0e00 x1862022114828800/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775768061.962945:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775768061.962947:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775768061.962948:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775768061.962952:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5439
00000100:00080000:0.0:1775768061.962955:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775768061.962957:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775768061.962963:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114828544:192.168.202.110@tcp:8:kworker.0
00000008:00100000:0.0:1775768061.962967:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775768061.962978:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3100 x1862022114828928/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775768061.962988:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114828672:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775768061.963008:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114828800:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775768061.963028:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114828928:192.168.202.110@tcp:8:kworker.0
00010000:00010000:0.0:1775768062.218748:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768062.986652:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768063.242690:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768064.010646:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775768064.266727:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768065.034626:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000400:00000100:1.0:1775768065.034767:0:2390:0:(lib-socket.c:454:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.202.110:988
00000400:00000100:1.0:1775768065.034798:0:2390:0:(acceptor.c:111:lnet_connect_console_error()) Connection to 192.168.202.110@tcp at host 192.168.202.110 was unreachable: the network or that node may be down, or Lustre may be misconfigured.
00000800:00000100:1.0:1775768065.034812:0:2390:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:1.0:1775768065.034851:0:2390:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:1.0:1775768065.034860:0:2390:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:1.0:1775768065.034867:0:2390:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:1.0:1775768065.034874:0:2390:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:1.0:1775768065.034882:0:2390:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:1.0:1775768065.034891:0:2390:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:1.0:1775768065.034898:0:2390:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000100:00100000:3.0:1775768065.034969:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768061/real 1775768065]  req@ffff9f14a04d3100 x1862022114828928/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768086 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775768065.034990:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3100 x1862022114828928/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768086 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775768065.035001:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775768065.035006:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775768065.035010:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114828928:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775768065.035019:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768061/real 1775768065]  req@ffff9f14a04d3800 x1862022114828544/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768086 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775768065.035034:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3800 x1862022114828544/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768086 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775768065.035042:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775768065.035045:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775768065.035048:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114828544:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775768065.035054:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768061/real 1775768065]  req@ffff9f14a04d1500 x1862022114828672/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768086 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775768065.035065:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1500 x1862022114828672/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768086 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775768065.035072:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775768065.035076:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775768065.035079:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114828672:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775768065.035086:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768061/real 1775768065]  req@ffff9f14a04d0e00 x1862022114828800/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768086 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775768065.035098:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0e00 x1862022114828800/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768086 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775768065.035106:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775768065.035110:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775768065.035112:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114828800:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775768065.035118:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768061/real 1775768065]  req@ffff9f14a04d0a80 x1862022114828160/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768086 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775768065.035128:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0a80 x1862022114828160/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768086 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775768065.035135:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775768065.035139:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775768065.035141:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114828160:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775768065.035147:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768061/real 1775768065]  req@ffff9f14a04d2300 x1862022114828288/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768086 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775768065.035157:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d2300 x1862022114828288/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768086 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775768065.035164:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775768065.035167:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775768065.035170:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114828288:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775768065.035175:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768061/real 1775768065]  req@ffff9f14a04d2a00 x1862022114828416/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768086 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775768065.035185:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d2a00 x1862022114828416/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768086 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775768065.035191:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775768065.035194:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775768065.035197:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114828416:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775768065.035203:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768061/real 1775768065]  req@ffff9f14a04d0000 x1862022114828032/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768086 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775768065.035212:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0000 x1862022114828032/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768086 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775768065.035218:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775768065.035221:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775768065.035224:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114828032:192.168.202.110@tcp:250:kworker.0
00010000:00010000:2.0:1775768065.290691:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768066.058665:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768066.314711:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768067.082618:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:0.0:1775768067.082625:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775768067.082630:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1775768067.082634:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775768067.082642:0:8:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5444
00000100:00080000:0.0:1775768067.082651:0:8:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775768067.082657:0:8:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775768067.082689:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d1180 x1862022114829056/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775768067.082716:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775768067.082724:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775768067.082726:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775768067.082730:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5444
00000100:00080000:0.0:1775768067.082734:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775768067.082738:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775768067.082749:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775768067.082757:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114829056:192.168.202.110@tcp:250:kworker.0
00000100:00100000:0.0:1775768067.082764:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3480 x1862022114829184/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775768067.082781:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775768067.082784:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:0.0:1775768067.082786:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775768067.082790:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5444
00000100:00080000:0.0:1775768067.082793:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775768067.082797:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775768067.082802:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775768067.082814:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0380 x1862022114829312/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:3.0:1775768067.082820:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:0.0:1775768067.082832:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775768067.082836:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775768067.082837:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775768067.082841:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5444
00000100:00080000:0.0:1775768067.082845:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775768067.082848:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775768067.082853:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775768067.082857:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768067/real 1775768067]  req@ffff9f14a04d1180 x1862022114829056/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768092 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768067.082868:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d1f80 x1862022114829440/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775768067.082878:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:3.0:1775768067.082879:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1180 x1862022114829056/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768092 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768067.082885:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1775768067.082887:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775768067.082893:0:8:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5444
00000100:00080000:0.0:1775768067.082897:0:8:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768067.082902:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768067.082903:0:8:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775768067.082911:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00080000:3.0:1775768067.082912:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775768067.082919:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114829056:192.168.202.110@tcp:250:kworker.0
00000100:00100000:0.0:1775768067.082927:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3b80 x1862022114829568/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775768067.082936:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775768067.082939:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775768067.082941:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:3.0:1775768067.082946:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114829184:192.168.202.110@tcp:38:kworker.0
00000100:00080000:0.0:1775768067.082959:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5444
00000100:00080000:0.0:1775768067.082964:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775768067.082968:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000800:00000100:3.0:1775768067.082972:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000008:00100000:0.0:1775768067.082974:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775768067.082986:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768067/real 1775768067]  req@ffff9f14a04d3480 x1862022114829184/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768092 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768067.082989:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d1180 x1862022114829696/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775768067.083014:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:3.0:1775768067.083015:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3480 x1862022114829184/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768092 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768067.083019:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:0.0:1775768067.083023:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775768067.083032:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5444
00000100:00080000:3.0:1775768067.083034:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768067.083036:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768067.083040:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00080000:0.0:1775768067.083042:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775768067.083047:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114829184:192.168.202.110@tcp:38:kworker.0
00000008:00100000:0.0:1775768067.083049:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775768067.083054:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114829312:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768067.083064:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0000 x1862022114829824/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:3.0:1775768067.083075:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:0.0:1775768067.083085:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:3.0:1775768067.083088:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768067/real 1775768067]  req@ffff9f14a04d0380 x1862022114829312/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768092 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768067.083090:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775768067.083094:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775768067.083104:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5444
00000100:00080000:0.0:1775768067.083109:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:3.0:1775768067.083109:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0380 x1862022114829312/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768092 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768067.083116:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775768067.083125:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00080000:3.0:1775768067.083126:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775768067.083133:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775768067.083136:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114829312:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768067.083142:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d2a00 x1862022114829952/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775768067.083144:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114829440:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775768067.083171:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775768067.083183:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768067/real 1775768067]  req@ffff9f14a04d1f80 x1862022114829440/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768092 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775768067.083196:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1f80 x1862022114829440/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768092 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775768067.083204:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775768067.083207:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775768067.083211:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114829440:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775768067.083216:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114829568:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775768067.083232:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775768067.083239:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768067/real 1775768067]  req@ffff9f14a04d3b80 x1862022114829568/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768092 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775768067.083250:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3b80 x1862022114829568/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768092 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775768067.083258:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775768067.083261:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775768067.083264:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114829568:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775768067.083349:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114829696:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775768067.083367:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775768067.083376:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768067/real 1775768067]  req@ffff9f14a04d1180 x1862022114829696/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768092 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775768067.083387:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1180 x1862022114829696/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768092 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775768067.083396:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775768067.083400:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775768067.083403:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114829696:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775768067.083408:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114829824:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775768067.083421:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775768067.083429:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768067/real 1775768067]  req@ffff9f14a04d0000 x1862022114829824/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768092 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775768067.083440:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0000 x1862022114829824/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768092 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775768067.083447:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775768067.083450:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775768067.083453:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114829824:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775768067.083458:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114829952:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775768067.083498:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775768067.083511:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768067/real 1775768067]  req@ffff9f14a04d2a00 x1862022114829952/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768092 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775768067.083523:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d2a00 x1862022114829952/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768092 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775768067.083531:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775768067.083534:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775768067.083537:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114829952:192.168.202.110@tcp:8:kworker.0
00010000:00010000:3.0:1775768067.338719:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768068.106658:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768068.362736:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768069.130663:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:0.0:1775768069.386691:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775768070.154685:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768070.410709:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775768071.178640:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775768071.434730:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1775768072.202653:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775768072.202664:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1775768072.202669:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775768072.202680:0:8:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5449
00000100:00100000:2.0:1775768072.202683:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:0.0:1775768072.202692:0:8:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775768072.202701:0:8:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775768072.202742:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d2300 x1862022114830080/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775768072.202764:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775768072.202766:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775768072.202767:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775768072.202772:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5449
00000100:00080000:0.0:1775768072.202775:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775768072.202778:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775768072.202786:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775768072.202795:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0a80 x1862022114830208/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775768072.202803:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775768072.202806:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:0.0:1775768072.202807:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775768072.202811:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5449
00000100:00100000:3.0:1775768072.202812:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114830080:192.168.202.110@tcp:250:kworker.0
00000100:00080000:0.0:1775768072.202813:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775768072.202819:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775768072.202823:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775768072.202831:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0e00 x1862022114830336/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775768072.202837:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775768072.202839:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775768072.202841:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775768072.202843:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5449
00000100:00080000:0.0:1775768072.202845:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775768072.202848:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775768072.202851:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775768072.202857:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d1500 x1862022114830464/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775768072.202864:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775768072.202866:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1775768072.202867:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775768072.202869:0:8:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5449
00000100:00080000:0.0:1775768072.202872:0:8:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775768072.202874:0:8:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775768072.202880:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:3.0:1775768072.202880:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768072.202890:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3800 x1862022114830592/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775768072.202897:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775768072.202900:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775768072.202901:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775768072.202904:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5449
00000100:00080000:0.0:1775768072.202906:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775768072.202908:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775768072.202911:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775768072.202918:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3100 x1862022114830720/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775768072.202924:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775768072.202926:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:0.0:1775768072.202928:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:3.0:1775768072.202930:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768072/real 1775768072]  req@ffff9f14a04d2300 x1862022114830080/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768097 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768072.202933:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5449
00000100:00080000:0.0:1775768072.202938:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775768072.202943:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775768072.202947:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775768072.202955:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d2a00 x1862022114830848/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775768072.202986:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775768072.202989:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775768072.202991:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775768072.202994:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5449
00000100:00080000:0.0:1775768072.202997:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:3.0:1775768072.203000:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d2300 x1862022114830080/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768097 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768072.203004:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775768072.203009:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00080000:3.0:1775768072.203019:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00100000:0.0:1775768072.203023:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0000 x1862022114830976/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775768072.203027:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775768072.203042:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114830080:192.168.202.110@tcp:250:kworker.0
00000100:00100000:3.0:1775768072.203063:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114830208:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775768072.203082:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775768072.203099:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768072/real 1775768072]  req@ffff9f14a04d0a80 x1862022114830208/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768097 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775768072.203113:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0a80 x1862022114830208/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768097 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775768072.203122:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775768072.203125:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775768072.203129:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114830208:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775768072.203136:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114830336:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775768072.203157:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775768072.203168:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768072/real 1775768072]  req@ffff9f14a04d0e00 x1862022114830336/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768097 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775768072.203180:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0e00 x1862022114830336/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768097 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775768072.203190:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775768072.203193:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775768072.203195:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114830336:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775768072.203199:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114830464:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775768072.203219:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775768072.203231:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768072/real 1775768072]  req@ffff9f14a04d1500 x1862022114830464/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768097 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775768072.203243:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1500 x1862022114830464/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768097 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775768072.203252:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775768072.203256:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775768072.203259:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114830464:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775768072.203265:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114830592:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775768072.203290:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775768072.203298:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768072/real 1775768072]  req@ffff9f14a04d3800 x1862022114830592/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768097 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775768072.203306:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3800 x1862022114830592/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768097 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775768072.203311:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775768072.203313:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775768072.203315:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114830592:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775768072.203319:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114830720:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775768072.203329:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775768072.203335:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768072/real 1775768072]  req@ffff9f14a04d3100 x1862022114830720/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768097 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775768072.203342:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3100 x1862022114830720/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768097 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775768072.203347:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775768072.203348:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775768072.203350:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114830720:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775768072.203354:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114830848:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775768072.203363:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775768072.203368:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768072/real 1775768072]  req@ffff9f14a04d2a00 x1862022114830848/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768097 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775768072.203375:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d2a00 x1862022114830848/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768097 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775768072.203380:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775768072.203382:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775768072.203383:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114830848:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775768072.203387:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114830976:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775768072.203396:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775768072.203401:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768072/real 1775768072]  req@ffff9f14a04d0000 x1862022114830976/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768097 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775768072.203408:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0000 x1862022114830976/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768097 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775768072.203413:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775768072.203415:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775768072.203416:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114830976:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775768072.458746:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768073.226612:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768073.482711:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768074.250651:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775768074.506752:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768075.274662:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768075.530702:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768076.298661:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:0.0:1775768076.554671:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768077.322653:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:0.0:1775768077.322659:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775768077.322666:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1775768077.322671:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775768077.322694:0:8:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5454
00000100:00080000:0.0:1775768077.322707:0:8:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775768077.322714:0:8:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775768077.322756:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d1180 x1862022114831104/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775768077.322787:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775768077.322791:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775768077.322792:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775768077.322796:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5454
00000100:00080000:0.0:1775768077.322799:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775768077.322802:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775768077.322812:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775768077.322825:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3b80 x1862022114831232/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775768077.322836:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114831104:192.168.202.110@tcp:250:kworker.0
00000100:00080000:0.0:1775768077.322840:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775768077.322855:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:0.0:1775768077.322858:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775768077.322864:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5454
00000100:00080000:0.0:1775768077.322868:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775768077.322873:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775768077.322878:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775768077.322889:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d1f80 x1862022114831360/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775768077.322896:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775768077.322899:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775768077.322900:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775768077.322905:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5454
00000800:00000100:3.0:1775768077.322909:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:0.0:1775768077.322910:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775768077.322915:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775768077.322920:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775768077.322928:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0380 x1862022114831488/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775768077.322935:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775768077.322937:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1775768077.322938:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775768077.322940:0:8:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5454
00000100:00080000:0.0:1775768077.322943:0:8:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775768077.322945:0:8:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775768077.322949:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775768077.322959:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3480 x1862022114831616/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775768077.322960:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768077/real 1775768077]  req@ffff9f14a04d1180 x1862022114831104/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768102 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768077.322975:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775768077.322982:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775768077.322985:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775768077.322992:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5454
00000100:00080000:0.0:1775768077.322994:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775768077.322997:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775768077.323000:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775768077.323005:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1180 x1862022114831104/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768102 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768077.323010:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0000 x1862022114831744/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775768077.323034:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775768077.323037:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775768077.323039:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768077.323040:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768077.323050:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775768077.323056:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114831104:192.168.202.110@tcp:250:kworker.0
00000100:00080000:0.0:1775768077.323057:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5454
00000100:00080000:0.0:1775768077.323062:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775768077.323066:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775768077.323069:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775768077.323076:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d2a00 x1862022114831872/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775768077.323082:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775768077.323085:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775768077.323086:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775768077.323088:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5454
00000100:00080000:0.0:1775768077.323090:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775768077.323093:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775768077.323096:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775768077.323098:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114831232:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768077.323105:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3100 x1862022114832000/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:3.0:1775768077.323133:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775768077.323150:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768077/real 1775768077]  req@ffff9f14a04d3b80 x1862022114831232/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768102 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775768077.323171:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3b80 x1862022114831232/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768102 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775768077.323180:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775768077.323184:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775768077.323187:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114831232:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775768077.323192:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114831360:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775768077.323208:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775768077.323216:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768077/real 1775768077]  req@ffff9f14a04d1f80 x1862022114831360/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768102 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775768077.323224:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1f80 x1862022114831360/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768102 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775768077.323229:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775768077.323231:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775768077.323233:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114831360:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775768077.323236:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114831488:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775768077.323251:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775768077.323260:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768077/real 1775768077]  req@ffff9f14a04d0380 x1862022114831488/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768102 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775768077.323272:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0380 x1862022114831488/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768102 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775768077.323287:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775768077.323290:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775768077.323294:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114831488:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775768077.323300:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114831616:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775768077.323318:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775768077.323325:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768077/real 1775768077]  req@ffff9f14a04d3480 x1862022114831616/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768102 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775768077.323333:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3480 x1862022114831616/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768102 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775768077.323339:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775768077.323341:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775768077.323342:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114831616:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775768077.323346:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114831744:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775768077.323356:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775768077.323362:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768077/real 1775768077]  req@ffff9f14a04d0000 x1862022114831744/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768102 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775768077.323369:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0000 x1862022114831744/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768102 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775768077.323399:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775768077.323401:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775768077.323404:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114831744:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775768077.323408:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114831872:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775768077.323423:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775768077.323430:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768077/real 1775768077]  req@ffff9f14a04d2a00 x1862022114831872/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768102 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775768077.323438:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d2a00 x1862022114831872/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768102 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775768077.323443:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775768077.323445:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775768077.323447:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114831872:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775768077.323462:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114832000:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775768077.323471:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775768077.323477:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768077/real 1775768077]  req@ffff9f14a04d3100 x1862022114832000/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768102 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775768077.323484:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3100 x1862022114832000/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768102 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775768077.323489:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775768077.323491:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775768077.323492:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114832000:192.168.202.110@tcp:8:kworker.0
00010000:00010000:3.0:1775768077.578707:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768078.346652:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775768078.602708:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768079.370662:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775768079.626738:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775768080.394714:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768080.650761:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775768081.418720:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775768081.674775:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775768082.442731:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775768082.442748:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775768082.442755:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775768082.442760:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768082.442771:0:8:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5459
00000100:00080000:3.0:1775768082.442786:0:8:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768082.442795:0:8:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775768082.442841:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744700 x1862022114832128/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775768082.442869:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775768082.442873:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775768082.442875:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768082.442880:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5459
00000100:00080000:3.0:1775768082.442884:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768082.442888:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775768082.442900:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775768082.442903:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114832128:192.168.202.110@tcp:250:kworker.0
00000100:00100000:3.0:1775768082.442916:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745f80 x1862022114832256/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775768082.442926:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775768082.442929:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775768082.442931:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768082.442935:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5459
00000100:00080000:3.0:1775768082.442938:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768082.442942:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775768082.442947:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775768082.442957:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747800 x1862022114832384/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775768082.442965:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775768082.442968:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775768082.442969:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768082.442973:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5459
00000100:00080000:3.0:1775768082.442976:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768082.442980:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000800:00000100:0.0:1775768082.442985:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000008:00100000:3.0:1775768082.442985:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775768082.442994:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744000 x1862022114832512/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775768082.443002:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775768082.443005:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775768082.443006:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768082.443010:0:8:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5459
00000100:00080000:3.0:1775768082.443013:0:8:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768082.443016:0:8:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775768082.443020:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775768082.443028:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747100 x1862022114832640/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775768082.443037:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1775768082.443040:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768082/real 1775768082]  req@ffff9f1485744700 x1862022114832128/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768107 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775768082.443040:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775768082.443042:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768082.443047:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5459
00000100:00080000:3.0:1775768082.443050:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768082.443054:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775768082.443059:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744700 x1862022114832128/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768107 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:3.0:1775768082.443059:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775768082.443067:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745180 x1862022114832768/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775768082.443071:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768082.443075:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:3.0:1775768082.443076:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1775768082.443080:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114832128:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775768082.443080:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775768082.443081:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768082.443085:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5459
00000100:00080000:3.0:1775768082.443088:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768082.443092:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775768082.443096:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775768082.443104:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114832256:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775768082.443105:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745c00 x1862022114832896/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775768082.443115:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775768082.443118:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775768082.443120:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768082.443124:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5459
00000800:00000100:0.0:1775768082.443127:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775768082.443127:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768082.443131:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775768082.443136:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775768082.443140:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768082/real 1775768082]  req@ffff9f1485745f80 x1862022114832256/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768107 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775768082.443146:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746680 x1862022114833024/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775768082.443153:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745f80 x1862022114832256/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768107 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768082.443161:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768082.443164:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768082.443167:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114832256:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768082.443173:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114832384:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768082.443194:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768082.443205:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768082/real 1775768082]  req@ffff9f1485747800 x1862022114832384/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768107 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768082.443213:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747800 x1862022114832384/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768107 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768082.443219:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768082.443221:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768082.443223:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114832384:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768082.443228:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114832512:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768082.443242:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768082.443248:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768082/real 1775768082]  req@ffff9f1485744000 x1862022114832512/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768107 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768082.443256:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744000 x1862022114832512/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768107 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768082.443261:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768082.443263:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768082.443265:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114832512:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775768082.443268:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114832640:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768082.443278:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768082.443283:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768082/real 1775768082]  req@ffff9f1485747100 x1862022114832640/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768107 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768082.443290:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747100 x1862022114832640/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768107 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768082.443295:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768082.443296:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768082.443298:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114832640:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775768082.443301:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114832768:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768082.443311:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768082.443316:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768082/real 1775768082]  req@ffff9f1485745180 x1862022114832768/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768107 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768082.443352:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745180 x1862022114832768/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768107 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768082.443358:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768082.443360:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768082.443362:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114832768:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768082.443376:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114832896:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768082.443390:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768082.443397:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768082/real 1775768082]  req@ffff9f1485745c00 x1862022114832896/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768107 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768082.443405:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745c00 x1862022114832896/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768107 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768082.443410:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768082.443413:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768082.443415:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114832896:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768082.443418:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114833024:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768082.443428:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768082.443433:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768082/real 1775768082]  req@ffff9f1485746680 x1862022114833024/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768107 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768082.443440:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746680 x1862022114833024/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768107 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768082.443450:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768082.443452:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768082.443455:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114833024:192.168.202.110@tcp:8:kworker.0
00010000:00010000:3.0:1775768082.698716:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775768083.466692:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:0.0:1775768083.722687:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775768084.490634:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768084.746691:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775768085.514657:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775768085.770720:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775768086.538657:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775768086.794707:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775768087.562663:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:1.0:1775768087.562677:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775768087.562684:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1775768087.562690:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775768087.562702:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5465
00000100:00080000:1.0:1775768087.562714:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775768087.562722:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775768087.562762:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fca80 x1862022114833152/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775768087.562786:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775768087.562788:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775768087.562790:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775768087.562793:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5465
00000100:00080000:1.0:1775768087.562796:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775768087.562800:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775768087.562810:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775768087.562817:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114833152:192.168.202.110@tcp:250:kworker.0
00000100:00100000:1.0:1775768087.562822:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ffb80 x1862022114833280/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775768087.562831:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775768087.562840:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775768087.562842:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775768087.562845:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5465
00000100:00080000:1.0:1775768087.562847:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775768087.562850:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775768087.562854:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775768087.562862:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fed80 x1862022114833408/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775768087.562869:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775768087.562871:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775768087.562872:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000800:00000100:0.0:1775768087.562877:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:1.0:1775768087.562877:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5465
00000100:00080000:1.0:1775768087.562883:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775768087.562886:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775768087.562890:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775768087.562898:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff800 x1862022114833536/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775768087.562904:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775768087.562907:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1775768087.562908:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775768087.562912:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5465
00000100:00100000:0.0:1775768087.562916:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768087/real 1775768087]  req@ffff9f14ab4fca80 x1862022114833152/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768112 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775768087.562917:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775768087.562923:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775768087.562928:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775768087.562940:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fc380 x1862022114833664/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775768087.562947:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1775768087.562951:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fca80 x1862022114833152/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768112 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775768087.562960:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775768087.562963:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775768087.562971:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5465
00000100:00080000:0.0:1775768087.562974:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775768087.562974:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775768087.562979:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775768087.562982:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00080000:0.0:1775768087.562983:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768087.562990:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114833152:192.168.202.110@tcp:250:kworker.0
00000100:00100000:1.0:1775768087.562993:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff100 x1862022114833792/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775768087.563002:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775768087.563004:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775768087.563006:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775768087.563009:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5465
00000100:00080000:1.0:1775768087.563011:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775768087.563013:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775768087.563017:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775768087.563022:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ffb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114833280:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775768087.563025:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fce00 x1862022114833920/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775768087.563035:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775768087.563037:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775768087.563039:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775768087.563041:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5465
00000100:00080000:1.0:1775768087.563044:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775768087.563046:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775768087.563050:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:0.0:1775768087.563059:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775768087.563060:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdc00 x1862022114834048/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775768087.563077:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768087/real 1775768087]  req@ffff9f14ab4ffb80 x1862022114833280/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768112 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768087.563092:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ffb80 x1862022114833280/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768112 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768087.563098:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768087.563101:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768087.563103:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ffb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114833280:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768087.563107:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114833408:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768087.563120:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768087.563128:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768087/real 1775768087]  req@ffff9f14ab4fed80 x1862022114833408/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768112 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768087.563140:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fed80 x1862022114833408/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768112 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768087.563148:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768087.563151:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768087.563155:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114833408:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768087.563161:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114833536:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768087.563182:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768087.563193:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768087/real 1775768087]  req@ffff9f14ab4ff800 x1862022114833536/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768112 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768087.563205:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff800 x1862022114833536/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768112 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768087.563216:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768087.563218:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768087.563220:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114833536:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775768087.563225:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114833664:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768087.563239:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768087.563246:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768087/real 1775768087]  req@ffff9f14ab4fc380 x1862022114833664/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768112 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768087.563253:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fc380 x1862022114833664/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768112 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768087.563258:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768087.563260:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768087.563262:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114833664:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775768087.563265:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114833792:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768087.563294:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768087.563306:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768087/real 1775768087]  req@ffff9f14ab4ff100 x1862022114833792/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768112 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768087.563318:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff100 x1862022114833792/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768112 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768087.563326:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768087.563329:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768087.563333:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114833792:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768087.563353:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114833920:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768087.563371:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768087.563379:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768087/real 1775768087]  req@ffff9f14ab4fce00 x1862022114833920/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768112 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768087.563386:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fce00 x1862022114833920/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768112 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768087.563391:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768087.563393:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768087.563395:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114833920:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768087.563399:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114834048:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768087.563409:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768087.563414:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768087/real 1775768087]  req@ffff9f14ab4fdc00 x1862022114834048/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768112 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768087.563421:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdc00 x1862022114834048/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768112 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768087.563425:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768087.563427:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768087.563429:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114834048:192.168.202.110@tcp:8:kworker.0
00010000:00010000:3.0:1775768087.818723:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775768088.586637:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768088.842707:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775768089.610676:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768089.866754:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775768090.634645:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:0.0:1775768090.890700:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775768091.658645:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768091.914719:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768092.682686:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:0.0:1775768092.683127:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775768092.683135:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1775768092.683139:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775768092.683149:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5470
00000100:00080000:0.0:1775768092.683169:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775768092.683177:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775768092.683215:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3800 x1862022114834176/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775768092.683269:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775768092.683273:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775768092.683275:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775768092.683280:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5470
00000100:00080000:0.0:1775768092.683285:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775768092.683290:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775768092.683301:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775768092.683313:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114834176:192.168.202.110@tcp:250:kworker.0
00000100:00100000:0.0:1775768092.683319:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d1500 x1862022114834304/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775768092.683332:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775768092.683336:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:0.0:1775768092.683338:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775768092.683343:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5470
00000100:00080000:0.0:1775768092.683347:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775768092.683352:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775768092.683358:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775768092.683370:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0e00 x1862022114834432/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:2.0:1775768092.683377:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:0.0:1775768092.683388:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775768092.683392:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775768092.683394:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775768092.683399:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5470
00000100:00080000:0.0:1775768092.683404:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775768092.683409:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775768092.683414:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768092/real 1775768092]  req@ffff9f14a04d3800 x1862022114834176/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768117 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:0.0:1775768092.683415:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775768092.683433:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0a80 x1862022114834560/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775768092.683443:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3800 x1862022114834176/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768117 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768092.683445:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775768092.683456:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1775768092.683460:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768092.683465:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768092.683467:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5470
00000100:00080000:0.0:1775768092.683473:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768092.683474:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:0.0:1775768092.683480:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775768092.683483:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114834176:192.168.202.110@tcp:250:kworker.0
00000008:00100000:0.0:1775768092.683487:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775768092.683502:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d2300 x1862022114834688/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775768092.683512:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114834304:192.168.202.110@tcp:38:kworker.0
00000100:00080000:0.0:1775768092.683513:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775768092.683522:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775768092.683524:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775768092.683530:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5470
00000100:00080000:0.0:1775768092.683535:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775768092.683540:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775768092.683546:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000800:00000100:2.0:1775768092.683547:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775768092.683600:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768092/real 1775768092]  req@ffff9f14a04d1500 x1862022114834304/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768117 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768092.683608:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3800 x1862022114834816/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775768092.683615:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1500 x1862022114834304/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768117 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768092.683635:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768092.683636:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775768092.683642:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00080000:0.0:1775768092.683644:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00100000:2.0:1775768092.683647:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114834304:192.168.202.110@tcp:38:kworker.0
00000100:00080000:0.0:1775768092.683648:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775768092.683659:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5470
00000100:00100000:2.0:1775768092.683660:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114834432:192.168.202.110@tcp:38:kworker.0
00000100:00080000:0.0:1775768092.683667:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775768092.683672:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775768092.683680:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000800:00000100:2.0:1775768092.683692:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768092.683698:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3100 x1862022114834944/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775768092.683703:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768092/real 1775768092]  req@ffff9f14a04d0e00 x1862022114834432/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768117 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768092.683724:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:2.0:1775768092.683728:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0e00 x1862022114834432/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768117 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768092.683732:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775768092.683737:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768092.683742:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768092.683746:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5470
00000100:00080000:2.0:1775768092.683749:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775768092.683754:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114834432:192.168.202.110@tcp:38:kworker.0
00000100:00080000:0.0:1775768092.683755:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:2.0:1775768092.683759:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114834560:192.168.202.110@tcp:8:kworker.0
00000100:00080000:0.0:1775768092.683763:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775768092.683771:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:2.0:1775768092.683777:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775768092.683788:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768092/real 1775768092]  req@ffff9f14a04d0a80 x1862022114834560/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768117 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768092.683789:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d2a00 x1862022114835072/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775768092.683812:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0a80 x1862022114834560/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768117 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775768092.683824:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775768092.683828:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775768092.683832:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114834560:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775768092.683849:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114834688:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775768092.683872:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775768092.683883:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768092/real 1775768092]  req@ffff9f14a04d2300 x1862022114834688/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768117 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775768092.683896:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d2300 x1862022114834688/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768117 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775768092.683902:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775768092.683912:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775768092.683915:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114834688:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775768092.683919:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114834816:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775768092.683934:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775768092.683941:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768092/real 1775768092]  req@ffff9f14a04d3800 x1862022114834816/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768117 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775768092.683949:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3800 x1862022114834816/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768117 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775768092.683954:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775768092.683956:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775768092.683958:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114834816:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775768092.683961:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114834944:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775768092.683971:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775768092.683979:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768092/real 1775768092]  req@ffff9f14a04d3100 x1862022114834944/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768117 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775768092.683990:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3100 x1862022114834944/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768117 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775768092.683998:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775768092.684011:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775768092.684015:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114834944:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775768092.684022:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114835072:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775768092.684043:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775768092.684053:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768092/real 1775768092]  req@ffff9f14a04d2a00 x1862022114835072/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768117 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775768092.684062:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d2a00 x1862022114835072/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768117 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775768092.684067:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775768092.684069:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775768092.684071:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114835072:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775768092.938730:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768093.706645:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775768093.962728:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768094.730661:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768094.986711:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768095.754689:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768096.010684:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768096.778653:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768097.034711:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1775768097.802659:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:2.0:1775768097.802667:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:0.0:1775768097.802670:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1775768097.802676:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775768097.802688:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5475
00000100:00080000:0.0:1775768097.802700:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775768097.802707:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775768097.802746:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0000 x1862022114835200/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775768097.802780:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775768097.802784:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775768097.802786:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775768097.802791:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5475
00000100:00080000:0.0:1775768097.802796:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775768097.802800:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775768097.802812:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775768097.802817:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114835200:192.168.202.110@tcp:250:kworker.0
00000100:00100000:0.0:1775768097.802829:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3480 x1862022114835328/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775768097.802843:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775768097.802846:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:0.0:1775768097.802849:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775768097.802861:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5475
00000100:00080000:0.0:1775768097.802866:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775768097.802871:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775768097.802877:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000800:00000100:2.0:1775768097.802885:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768097.802891:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0380 x1862022114835456/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775768097.802902:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775768097.802906:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775768097.802908:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775768097.802912:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5475
00000100:00080000:0.0:1775768097.802916:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775768097.802921:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775768097.802926:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775768097.802932:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768097/real 1775768097]  req@ffff9f14a04d0000 x1862022114835200/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768122 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768097.802939:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d1f80 x1862022114835584/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775768097.802959:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:2.0:1775768097.802963:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0000 x1862022114835200/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768122 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768097.802964:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1775768097.802968:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775768097.802978:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5475
00000100:00080000:2.0:1775768097.802985:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768097.802986:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768097.802993:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:0.0:1775768097.802994:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775768097.803000:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114835200:192.168.202.110@tcp:250:kworker.0
00000008:00100000:0.0:1775768097.803004:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775768097.803019:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3b80 x1862022114835712/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775768097.803022:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114835328:192.168.202.110@tcp:38:kworker.0
00000100:00080000:0.0:1775768097.803045:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000800:00000100:2.0:1775768097.803048:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:0.0:1775768097.803053:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775768097.803056:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:2.0:1775768097.803062:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768097/real 1775768097]  req@ffff9f14a04d3480 x1862022114835328/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768122 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768097.803064:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5475
00000100:00080000:0.0:1775768097.803073:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:2.0:1775768097.803077:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3480 x1862022114835328/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768122 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775768097.803083:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775768097.803086:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775768097.803089:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114835328:192.168.202.110@tcp:38:kworker.0
00000100:00080000:0.0:1775768097.803094:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775768097.803098:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114835456:192.168.202.110@tcp:38:kworker.0
00000008:00100000:0.0:1775768097.803104:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775768097.803121:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d1180 x1862022114835840/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:2.0:1775768097.803123:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775768097.803133:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768097/real 1775768097]  req@ffff9f14a04d0380 x1862022114835456/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768122 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768097.803136:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775768097.803142:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:0.0:1775768097.803146:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775768097.803153:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5475
00000100:00100000:2.0:1775768097.803154:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0380 x1862022114835456/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768122 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768097.803160:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768097.803162:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768097.803168:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00080000:2.0:1775768097.803168:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775768097.803188:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114835456:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775768097.803195:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114835584:192.168.202.110@tcp:8:kworker.0
00000008:00100000:0.0:1775768097.803206:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000800:00000100:2.0:1775768097.803221:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768097.803232:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0000 x1862022114835968/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775768097.803237:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768097/real 1775768097]  req@ffff9f14a04d1f80 x1862022114835584/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768122 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768097.803258:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775768097.803262:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00100000:2.0:1775768097.803265:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1f80 x1862022114835584/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768122 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768097.803266:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775768097.803276:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5475
00000100:00080000:2.0:1775768097.803281:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768097.803283:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768097.803288:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00080000:0.0:1775768097.803293:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775768097.803296:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114835584:192.168.202.110@tcp:8:kworker.0
00000008:00100000:0.0:1775768097.803302:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775768097.803319:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d2a00 x1862022114836096/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775768097.803321:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114835712:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775768097.803344:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775768097.803355:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768097/real 1775768097]  req@ffff9f14a04d3b80 x1862022114835712/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768122 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775768097.803364:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3b80 x1862022114835712/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768122 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775768097.803370:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775768097.803372:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775768097.803374:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114835712:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775768097.803377:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114835840:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775768097.803387:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775768097.803393:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768097/real 1775768097]  req@ffff9f14a04d1180 x1862022114835840/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768122 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775768097.803401:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1180 x1862022114835840/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768122 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775768097.803406:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775768097.803408:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775768097.803409:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114835840:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775768097.803413:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114835968:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775768097.803422:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775768097.803427:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768097/real 1775768097]  req@ffff9f14a04d0000 x1862022114835968/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768122 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775768097.803435:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0000 x1862022114835968/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768122 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775768097.803439:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775768097.803441:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775768097.803443:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114835968:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775768097.803452:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114836096:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775768097.803461:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775768097.803466:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768097/real 1775768097]  req@ffff9f14a04d2a00 x1862022114836096/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768122 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775768097.803473:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d2a00 x1862022114836096/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768122 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775768097.803482:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775768097.803484:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775768097.803486:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114836096:192.168.202.110@tcp:8:kworker.0
00010000:00010000:0.0:1775768098.058696:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768098.826692:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768099.082697:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768099.850641:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775768100.106735:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775768100.874705:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775768101.130756:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775768101.898733:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768102.154729:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1775768102.922742:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1775768102.922746:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:2.0:1775768102.922758:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1775768102.922763:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768102.922774:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5480
00000100:00080000:2.0:1775768102.922787:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768102.922796:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775768102.922846:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2d80 x1862022114836224/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768102.922881:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768102.922884:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775768102.922885:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768102.922888:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5480
00000100:00080000:2.0:1775768102.922891:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768102.922894:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768102.922910:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775768102.922921:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114836224:192.168.202.110@tcp:250:kworker.0
00000100:00100000:2.0:1775768102.922921:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3b80 x1862022114836352/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768102.922946:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768102.922948:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775768102.922949:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768102.922959:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5480
00000100:00080000:2.0:1775768102.922961:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768102.922965:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768102.922968:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775768102.922975:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0000 x1862022114836480/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768102.922980:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768102.922982:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775768102.922984:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768102.922986:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5480
00000100:00080000:2.0:1775768102.922987:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768102.922990:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768102.922992:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775768102.923013:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1180 x1862022114836608/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768102.923022:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768102.923025:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1775768102.923027:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768102.923031:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5480
00000100:00080000:2.0:1775768102.923036:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768102.923040:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768102.923045:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:0.0:1775768102.923048:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775768102.923071:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1880 x1862022114836736/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768102.923078:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768102.923081:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775768102.923082:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768102.923086:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5480
00000100:00080000:2.0:1775768102.923090:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768102.923093:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768102.923097:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775768102.923117:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768102/real 1775768102]  req@ffff9f14852d2d80 x1862022114836224/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768127 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775768102.923121:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0e00 x1862022114836864/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768102.923144:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768102.923163:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775768102.923165:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768102.923172:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5480
00000100:00100000:0.0:1775768102.923174:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2d80 x1862022114836224/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768127 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775768102.923177:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768102.923192:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00080000:0.0:1775768102.923200:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000008:00100000:2.0:1775768102.923201:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00080000:0.0:1775768102.923210:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768102.923217:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114836224:192.168.202.110@tcp:250:kworker.0
00000100:00100000:2.0:1775768102.923228:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2300 x1862022114836992/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768102.923239:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768102.923241:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775768102.923243:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768102.923248:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5480
00000100:00080000:2.0:1775768102.923250:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1775768102.923251:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114836352:192.168.202.110@tcp:38:kworker.0
00000100:00080000:2.0:1775768102.923265:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768102.923271:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775768102.923286:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3100 x1862022114837120/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1775768102.923292:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768102.923316:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768102/real 1775768102]  req@ffff9f14852d3b80 x1862022114836352/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768127 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768102.923334:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3b80 x1862022114836352/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768127 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768102.923349:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768102.923353:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768102.923357:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114836352:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768102.923362:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114836480:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768102.923381:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768102.923389:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768102/real 1775768102]  req@ffff9f14852d0000 x1862022114836480/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768127 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768102.923397:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0000 x1862022114836480/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768127 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768102.923403:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768102.923405:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768102.923407:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114836480:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768102.923411:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114836608:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768102.923423:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768102.923428:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768102/real 1775768102]  req@ffff9f14852d1180 x1862022114836608/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768127 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768102.923436:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1180 x1862022114836608/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768127 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768102.923441:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768102.923443:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768102.923446:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114836608:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775768102.923450:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114836736:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768102.923460:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768102.923466:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768102/real 1775768102]  req@ffff9f14852d1880 x1862022114836736/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768127 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768102.923473:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1880 x1862022114836736/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768127 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768102.923478:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768102.923480:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768102.923482:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114836736:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775768102.923485:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114836864:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768102.923495:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768102.923501:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768102/real 1775768102]  req@ffff9f14852d0e00 x1862022114836864/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768127 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768102.923508:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0e00 x1862022114836864/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768127 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768102.923513:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768102.923515:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768102.923517:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114836864:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768102.923520:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114836992:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768102.923530:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768102.923536:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768102/real 1775768102]  req@ffff9f14852d2300 x1862022114836992/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768127 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768102.923543:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2300 x1862022114836992/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768127 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768102.923548:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768102.923550:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768102.923552:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114836992:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768102.923596:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114837120:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768102.923619:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768102.923630:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768102/real 1775768102]  req@ffff9f14852d3100 x1862022114837120/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768127 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768102.923642:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3100 x1862022114837120/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768127 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768102.923648:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768102.923650:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768102.923652:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114837120:192.168.202.110@tcp:8:kworker.0
00010000:00010000:3.0:1775768103.178785:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775768103.946687:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768104.202732:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775768104.970690:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:0.0:1775768105.226764:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775768105.994645:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768106.250740:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768107.018687:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775768107.274725:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768108.042638:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775768108.042643:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775768108.042649:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775768108.042653:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768108.042662:0:8:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5485
00000100:00080000:3.0:1775768108.042672:0:8:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768108.042678:0:8:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775768108.042712:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745500 x1862022114837248/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775768108.042734:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775768108.042737:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775768108.042738:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768108.042741:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5485
00000100:00080000:3.0:1775768108.042744:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768108.042747:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775768108.042754:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775768108.042762:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744e00 x1862022114837376/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775768108.042768:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775768108.042770:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775768108.042771:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768108.042774:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5485
00000100:00080000:3.0:1775768108.042775:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768108.042777:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775768108.042779:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775768108.042785:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744380 x1862022114837504/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775768108.042791:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775768108.042792:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775768108.042793:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768108.042796:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5485
00000100:00080000:3.0:1775768108.042798:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768108.042801:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775768108.042804:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775768108.042810:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746a00 x1862022114837632/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775768108.042815:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114837248:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775768108.042822:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775768108.042825:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775768108.042827:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768108.042829:0:8:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5485
00000100:00080000:3.0:1775768108.042831:0:8:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768108.042833:0:8:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775768108.042836:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775768108.042843:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744a80 x1862022114837760/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775768108.042849:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775768108.042851:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775768108.042858:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768108.042861:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5485
00000100:00080000:3.0:1775768108.042864:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768108.042867:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775768108.042870:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000800:00000100:0.0:1775768108.042877:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775768108.042880:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747480 x1862022114837888/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775768108.042888:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775768108.042890:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775768108.042891:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768108.042894:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5485
00000100:00080000:3.0:1775768108.042896:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768108.042898:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775768108.042902:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775768108.042910:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745880 x1862022114838016/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775768108.042916:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768108/real 1775768108]  req@ffff9f1485745500 x1862022114837248/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768133 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775768108.042918:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775768108.042929:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775768108.042931:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768108.042935:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5485
00000100:00080000:3.0:1775768108.042938:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1775768108.042939:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745500 x1862022114837248/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768133 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775768108.042943:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775768108.042951:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00080000:0.0:1775768108.042959:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00100000:3.0:1775768108.042962:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746300 x1862022114838144/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775768108.042969:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768108.042977:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114837248:192.168.202.110@tcp:250:kworker.0
00000100:00100000:0.0:1775768108.042987:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114837376:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768108.043012:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768108.043025:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768108/real 1775768108]  req@ffff9f1485744e00 x1862022114837376/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768133 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768108.043034:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744e00 x1862022114837376/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768133 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768108.043040:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768108.043042:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768108.043044:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114837376:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768108.043048:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114837504:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768108.043061:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768108.043067:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768108/real 1775768108]  req@ffff9f1485744380 x1862022114837504/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768133 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768108.043087:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744380 x1862022114837504/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768133 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768108.043103:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768108.043106:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768108.043109:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114837504:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768108.043129:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114837632:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768108.043144:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768108.043151:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768108/real 1775768108]  req@ffff9f1485746a00 x1862022114837632/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768133 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768108.043158:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746a00 x1862022114837632/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768133 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768108.043164:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768108.043165:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768108.043167:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114837632:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775768108.043171:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114837760:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768108.043180:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768108.043185:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768108/real 1775768108]  req@ffff9f1485744a80 x1862022114837760/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768133 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768108.043192:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744a80 x1862022114837760/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768133 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768108.043196:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768108.043198:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768108.043200:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114837760:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775768108.043204:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114837888:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768108.043213:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768108.043218:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768108/real 1775768108]  req@ffff9f1485747480 x1862022114837888/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768133 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768108.043224:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747480 x1862022114837888/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768133 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768108.043230:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768108.043231:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768108.043233:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114837888:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768108.043236:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114838016:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768108.043244:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768108.043249:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768108/real 1775768108]  req@ffff9f1485745880 x1862022114838016/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768133 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768108.043256:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745880 x1862022114838016/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768133 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768108.043261:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768108.043263:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768108.043264:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114838016:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768108.043268:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114838144:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768108.043276:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768108.043282:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768108/real 1775768108]  req@ffff9f1485746300 x1862022114838144/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768133 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768108.043289:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746300 x1862022114838144/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768133 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768108.043294:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768108.043296:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768108.043297:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114838144:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775768108.298728:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768109.066674:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768109.322767:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768110.090632:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768110.346711:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775768111.114694:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768111.370753:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775768112.138638:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:0.0:1775768112.394700:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1775768113.162635:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775768113.162654:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775768113.162658:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00100000:2.0:1775768113.162663:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775768113.162668:0:8:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5490
00000100:00080000:3.0:1775768113.162680:0:8:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768113.162687:0:8:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775768113.162734:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747b80 x1862022114838272/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775768113.162756:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775768113.162759:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775768113.162760:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768113.162763:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5490
00000100:00080000:3.0:1775768113.162767:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768113.162771:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775768113.162779:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775768113.162787:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114838272:192.168.202.110@tcp:250:kworker.0
00000100:00100000:3.0:1775768113.162792:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746300 x1862022114838400/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775768113.162801:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775768113.162803:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775768113.162805:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768113.162808:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5490
00000100:00080000:3.0:1775768113.162811:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768113.162813:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775768113.162816:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775768113.162823:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745880 x1862022114838528/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775768113.162829:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775768113.162831:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775768113.162832:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768113.162835:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5490
00000100:00080000:3.0:1775768113.162838:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768113.162840:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775768113.162845:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:0.0:1775768113.162846:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775768113.162853:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747480 x1862022114838656/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775768113.162861:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775768113.162863:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775768113.162864:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768113.162867:0:8:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5490
00000100:00080000:3.0:1775768113.162869:0:8:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768113.162871:0:8:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775768113.162874:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775768113.162882:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768113/real 1775768113]  req@ffff9f1485747b80 x1862022114838272/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768138 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775768113.162884:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744a80 x1862022114838784/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775768113.162914:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775768113.162918:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00100000:0.0:1775768113.162920:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747b80 x1862022114838272/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768138 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775768113.162921:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768113.162931:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5490
00000100:00080000:0.0:1775768113.162935:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775768113.162937:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775768113.162941:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:3.0:1775768113.162945:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775768113.162947:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114838272:192.168.202.110@tcp:250:kworker.0
00000008:00100000:3.0:1775768113.162954:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775768113.162967:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746a00 x1862022114838912/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775768113.162972:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114838400:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775768113.162981:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775768113.162984:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775768113.162985:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768113.162989:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5490
00000100:00080000:3.0:1775768113.162993:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768113.162996:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775768113.163000:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000800:00000100:0.0:1775768113.163006:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775768113.163020:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744380 x1862022114839040/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775768113.163038:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768113/real 1775768113]  req@ffff9f1485746300 x1862022114838400/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768138 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775768113.163043:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775768113.163047:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775768113.163048:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768113.163055:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5490
00000100:00080000:3.0:1775768113.163062:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768113.163070:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775768113.163077:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775768113.163078:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746300 x1862022114838400/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768138 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768113.163087:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768113.163092:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775768113.163092:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744e00 x1862022114839168/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775768113.163100:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114838400:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768113.163112:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114838528:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768113.163132:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768113.163140:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768113/real 1775768113]  req@ffff9f1485745880 x1862022114838528/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768138 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768113.163148:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745880 x1862022114838528/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768138 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768113.163153:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768113.163155:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768113.163157:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114838528:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768113.163161:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114838656:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768113.163172:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768113.163177:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768113/real 1775768113]  req@ffff9f1485747480 x1862022114838656/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768138 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768113.163184:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747480 x1862022114838656/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768138 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768113.163193:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768113.163196:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768113.163198:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114838656:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775768113.163201:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114838784:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768113.163210:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768113.163215:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768113/real 1775768113]  req@ffff9f1485744a80 x1862022114838784/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768138 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768113.163222:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744a80 x1862022114838784/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768138 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768113.163227:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768113.163229:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768113.163231:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114838784:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775768113.163242:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114838912:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768113.163251:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768113.163257:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768113/real 1775768113]  req@ffff9f1485746a00 x1862022114838912/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768138 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768113.163264:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746a00 x1862022114838912/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768138 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768113.163269:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768113.163271:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768113.163273:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114838912:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768113.163277:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114839040:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768113.163286:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768113.163291:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768113/real 1775768113]  req@ffff9f1485744380 x1862022114839040/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768138 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768113.163298:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744380 x1862022114839040/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768138 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768113.163302:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768113.163304:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768113.163306:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114839040:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768113.163309:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114839168:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768113.163318:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768113.163322:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768113/real 1775768113]  req@ffff9f1485744e00 x1862022114839168/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768138 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768113.163329:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744e00 x1862022114839168/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768138 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768113.163334:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768113.163336:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768113.163338:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114839168:192.168.202.110@tcp:8:kworker.0
00010000:00010000:3.0:1775768113.418735:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768114.186636:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775768114.442736:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768115.210669:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775768115.466755:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768116.234682:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768116.490723:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768117.258614:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768117.514740:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1775768118.282671:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775768118.282680:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00100000:2.0:1775768118.282685:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775768118.282685:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768118.282696:0:8:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5495
00000100:00080000:3.0:1775768118.282707:0:8:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768118.282714:0:8:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775768118.282746:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745500 x1862022114839296/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775768118.282774:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775768118.282777:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775768118.282778:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768118.282781:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5495
00000100:00080000:3.0:1775768118.282783:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768118.282785:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775768118.282793:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775768118.282800:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746680 x1862022114839424/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775768118.282806:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775768118.282809:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775768118.282810:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768118.282813:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5495
00000100:00080000:3.0:1775768118.282814:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768118.282816:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775768118.282819:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775768118.282824:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745c00 x1862022114839552/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775768118.282831:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775768118.282834:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00100000:0.0:1775768118.282835:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114839296:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775768118.282835:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768118.282840:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5495
00000100:00080000:3.0:1775768118.282845:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768118.282851:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775768118.282855:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775768118.282863:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745180 x1862022114839680/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775768118.282868:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775768118.282870:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775768118.282871:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768118.282873:0:8:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5495
00000100:00080000:3.0:1775768118.282875:0:8:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768118.282877:0:8:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775768118.282879:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775768118.282884:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747100 x1862022114839808/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775768118.282889:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775768118.282892:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775768118.282893:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768118.282895:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5495
00000100:00080000:3.0:1775768118.282898:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768118.282900:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775768118.282902:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000800:00000100:0.0:1775768118.282907:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775768118.282910:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744000 x1862022114839936/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775768118.282917:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775768118.282919:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775768118.282920:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768118.282923:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5495
00000100:00080000:3.0:1775768118.282925:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768118.282927:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775768118.282930:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775768118.282937:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747800 x1862022114840064/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775768118.282941:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775768118.282943:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775768118.282944:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768118.282947:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5495
00000100:00080000:3.0:1775768118.282949:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768118.282951:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775768118.282955:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775768118.282957:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768118/real 1775768118]  req@ffff9f1485745500 x1862022114839296/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768143 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775768118.282968:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745f80 x1862022114840192/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775768118.283008:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745500 x1862022114839296/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768143 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768118.283020:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768118.283024:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768118.283028:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114839296:192.168.202.110@tcp:250:kworker.0
00000100:00100000:0.0:1775768118.283034:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114839424:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768118.283054:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768118.283064:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768118/real 1775768118]  req@ffff9f1485746680 x1862022114839424/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768143 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768118.283075:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746680 x1862022114839424/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768143 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768118.283084:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768118.283087:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768118.283090:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114839424:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768118.283122:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114839552:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768118.283142:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768118.283152:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768118/real 1775768118]  req@ffff9f1485745c00 x1862022114839552/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768143 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768118.283163:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745c00 x1862022114839552/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768143 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768118.283174:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768118.283177:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768118.283180:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114839552:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768118.283186:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114839680:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768118.283202:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768118.283211:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768118/real 1775768118]  req@ffff9f1485745180 x1862022114839680/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768143 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768118.283221:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745180 x1862022114839680/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768143 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768118.283229:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768118.283232:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768118.283235:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114839680:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775768118.283240:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114839808:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768118.283254:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768118.283263:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768118/real 1775768118]  req@ffff9f1485747100 x1862022114839808/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768143 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768118.283273:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747100 x1862022114839808/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768143 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768118.283280:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768118.283283:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768118.283287:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114839808:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775768118.283293:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114839936:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768118.283307:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768118.283315:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768118/real 1775768118]  req@ffff9f1485744000 x1862022114839936/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768143 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768118.283325:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744000 x1862022114839936/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768143 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768118.283332:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768118.283335:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768118.283338:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114839936:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768118.283343:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114840064:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768118.283358:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768118.283366:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768118/real 1775768118]  req@ffff9f1485747800 x1862022114840064/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768143 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768118.283377:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747800 x1862022114840064/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768143 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768118.283385:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768118.283388:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768118.283390:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114840064:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768118.283396:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114840192:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768118.283420:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768118.283431:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768118/real 1775768118]  req@ffff9f1485745f80 x1862022114840192/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768143 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768118.283441:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745f80 x1862022114840192/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768143 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768118.283448:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768118.283451:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768118.283454:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114840192:192.168.202.110@tcp:8:kworker.0
00010000:00010000:3.0:1775768118.538724:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768119.306649:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:0.0:1775768119.562823:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768120.330646:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768120.586748:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775768121.354677:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775768121.610683:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775768122.378669:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775768122.634704:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775768123.402605:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:2.0:1775768123.402658:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768123.402665:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1775768123.402669:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768123.402678:0:8:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5500
00000100:00080000:2.0:1775768123.402687:0:8:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768123.402693:0:8:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775768123.402727:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1c00 x1862022114840320/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768123.402759:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768123.402762:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775768123.402764:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768123.402768:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5500
00000100:00080000:2.0:1775768123.402771:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768123.402775:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775768123.402782:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114840320:192.168.202.110@tcp:250:kworker.0
00000008:00100000:2.0:1775768123.402785:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775768123.402799:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3480 x1862022114840448/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768123.402810:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768123.402813:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775768123.402815:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768123.402819:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5500
00000100:00080000:2.0:1775768123.402823:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768123.402826:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768123.402831:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000800:00000100:0.0:1775768123.402840:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775768123.402844:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3100 x1862022114840576/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768123.402854:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768123.402857:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775768123.402859:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768123.402863:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5500
00000100:00080000:2.0:1775768123.402866:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768123.402870:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775768123.402873:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768123/real 1775768123]  req@ffff9f14852d1c00 x1862022114840320/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768148 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:2.0:1775768123.402876:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775768123.402889:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1c00 x1862022114840320/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768148 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775768123.402892:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2300 x1862022114840704/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775768123.402904:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768123.402911:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:2.0:1775768123.402911:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1775768123.402917:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114840320:192.168.202.110@tcp:250:kworker.0
00000100:00080000:2.0:1775768123.402917:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1775768123.402922:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768123.402942:0:8:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5500
00000100:00080000:2.0:1775768123.402946:0:8:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768123.402950:0:8:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775768123.402953:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114840448:192.168.202.110@tcp:38:kworker.0
00000008:00100000:2.0:1775768123.402956:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775768123.402970:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0e00 x1862022114840832/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768123.402981:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768123.402985:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000800:00000100:0.0:1775768123.402986:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:2.0:1775768123.402988:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768123.402994:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5500
00000100:00080000:2.0:1775768123.402998:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1775768123.402999:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768123/real 1775768123]  req@ffff9f14852d3480 x1862022114840448/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768148 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775768123.403006:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768123.403013:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775768123.403014:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3480 x1862022114840448/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768148 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768123.403022:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768123.403025:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775768123.403028:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1880 x1862022114840960/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775768123.403030:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114840448:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768123.403039:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114840576:192.168.202.110@tcp:38:kworker.0
00000100:00080000:2.0:1775768123.403046:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768123.403049:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775768123.403051:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768123.403056:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5500
00000800:00000100:0.0:1775768123.403059:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:2.0:1775768123.403063:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768123.403068:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775768123.403069:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768123/real 1775768123]  req@ffff9f14852d3100 x1862022114840576/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768148 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:2.0:1775768123.403075:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775768123.403080:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3100 x1862022114840576/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768148 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768123.403086:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00100000:2.0:1775768123.403090:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1180 x1862022114841088/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775768123.403092:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768123.403098:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114840576:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768123.403108:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114840704:192.168.202.110@tcp:8:kworker.0
00000100:00080000:2.0:1775768123.403111:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768123.403115:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775768123.403117:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768123.403122:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5500
00000800:00000100:0.0:1775768123.403124:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:2.0:1775768123.403129:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768123.403135:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775768123.403136:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768123/real 1775768123]  req@ffff9f14852d2300 x1862022114840704/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768148 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:2.0:1775768123.403142:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775768123.403146:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2300 x1862022114840704/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768148 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775768123.403155:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0000 x1862022114841216/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775768123.403158:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768123.403164:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768123.403170:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114840704:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775768123.403187:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114840832:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768123.403199:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768123.403205:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768123/real 1775768123]  req@ffff9f14852d0e00 x1862022114840832/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768148 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768123.403211:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0e00 x1862022114840832/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768148 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768123.403215:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768123.403217:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768123.403218:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114840832:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775768123.403221:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114840960:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768123.403230:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768123.403234:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768123/real 1775768123]  req@ffff9f14852d1880 x1862022114840960/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768148 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768123.403241:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1880 x1862022114840960/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768148 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768123.403245:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768123.403246:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768123.403248:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114840960:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768123.403252:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114841088:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768123.403259:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768123.403263:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768123/real 1775768123]  req@ffff9f14852d1180 x1862022114841088/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768148 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768123.403269:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1180 x1862022114841088/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768148 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768123.403273:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768123.403275:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768123.403276:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114841088:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768123.403279:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114841216:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768123.403287:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768123.403291:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768123/real 1775768123]  req@ffff9f14852d0000 x1862022114841216/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768148 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768123.403297:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0000 x1862022114841216/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768148 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768123.403301:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768123.403302:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768123.403304:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114841216:192.168.202.110@tcp:8:kworker.0
00010000:00010000:3.0:1775768123.658676:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775768124.426673:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768124.682767:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775768125.450638:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768125.706666:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775768126.474694:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:0.0:1775768126.730674:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775768127.498665:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768127.754682:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775768128.522628:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:2.0:1775768128.522662:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768128.522667:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1775768128.522671:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768128.522678:0:8:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5506
00000100:00080000:2.0:1775768128.522687:0:8:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768128.522692:0:8:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775768128.522724:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3b80 x1862022114841344/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768128.522741:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768128.522744:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775768128.522745:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768128.522748:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5506
00000100:00080000:2.0:1775768128.522751:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768128.522755:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768128.522765:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775768128.522777:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2d80 x1862022114841472/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775768128.522780:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114841344:192.168.202.110@tcp:250:kworker.0
00000100:00080000:2.0:1775768128.522795:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768128.522798:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775768128.522799:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768128.522804:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5506
00000100:00080000:2.0:1775768128.522807:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768128.522810:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768128.522821:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775768128.522832:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0700 x1862022114841600/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768128.522839:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768128.522841:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775768128.522842:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768128.522844:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5506
00000100:00080000:2.0:1775768128.522846:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768128.522849:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768128.522852:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775768128.522863:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0380 x1862022114841728/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768128.522869:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768128.522871:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1775768128.522872:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768128.522875:0:8:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5506
00000100:00100000:0.0:1775768128.522892:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114841472:192.168.202.110@tcp:38:kworker.0
00000100:00080000:2.0:1775768128.522892:0:8:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768128.522898:0:8:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768128.522903:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775768128.522912:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2a00 x1862022114841856/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768128.522920:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768128.522922:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775768128.522928:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775768128.522929:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114841600:192.168.202.110@tcp:38:kworker.0
00000100:00080000:2.0:1775768128.522934:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5506
00000100:00080000:2.0:1775768128.522938:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768128.522941:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768128.522945:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775768128.522955:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1500 x1862022114841984/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775768128.522957:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114841728:192.168.202.110@tcp:8:kworker.0
00000100:00080000:2.0:1775768128.522970:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768128.522974:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775768128.522976:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768128.522982:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5506
00000100:00080000:2.0:1775768128.522987:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1775768128.522989:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114841856:192.168.202.110@tcp:8:kworker.0
00000100:00080000:2.0:1775768128.522994:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768128.523001:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775768128.523014:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114841984:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775768128.523016:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2680 x1862022114842112/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768128.523030:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768128.523034:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775768128.523036:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775768128.523040:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114842112:192.168.202.110@tcp:38:kworker.0
00000100:00080000:2.0:1775768128.523043:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5506
00000100:00080000:2.0:1775768128.523050:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768128.523054:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768128.523060:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775768128.523074:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3800 x1862022114842240/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775768128.523143:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114842240:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775768128.778719:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775768129.546629:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775768129.802689:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775768130.570653:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768130.826698:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775768131.594637:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000400:00000100:3.0:1775768131.594769:0:2392:0:(lib-socket.c:454:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.202.110:988
00000400:00000100:3.0:1775768131.594805:0:2392:0:(acceptor.c:111:lnet_connect_console_error()) Connection to 192.168.202.110@tcp at host 192.168.202.110 was unreachable: the network or that node may be down, or Lustre may be misconfigured.
00000800:00000100:3.0:1775768131.594818:0:2392:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:3.0:1775768131.594885:0:2392:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:3.0:1775768131.594896:0:2392:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:3.0:1775768131.594908:0:2392:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000100:00100000:0.0:1775768131.594915:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768128/real 1775768131]  req@ffff9f14852d2d80 x1862022114841472/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768153 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000800:00000100:3.0:1775768131.594920:0:2392:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:3.0:1775768131.594932:0:2392:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000100:00100000:0.0:1775768131.594945:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2d80 x1862022114841472/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768153 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000800:00000100:3.0:1775768131.594950:0:2392:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000100:00080000:0.0:1775768131.594959:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768131.594966:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768131.594973:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114841472:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775768131.594973:0:2392:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000100:00100000:0.0:1775768131.594985:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768128/real 1775768131]  req@ffff9f14852d0700 x1862022114841600/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768153 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768131.594998:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0700 x1862022114841600/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768153 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768131.595005:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768131.595008:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768131.595011:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114841600:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768131.595017:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768128/real 1775768131]  req@ffff9f14852d0380 x1862022114841728/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768153 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768131.595027:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0380 x1862022114841728/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768153 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768131.595034:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768131.595038:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768131.595040:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114841728:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775768131.595046:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768128/real 1775768131]  req@ffff9f14852d3b80 x1862022114841344/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768153 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768131.595056:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3b80 x1862022114841344/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768153 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768131.595063:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768131.595066:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768131.595069:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114841344:192.168.202.110@tcp:250:kworker.0
00000100:00100000:0.0:1775768131.595093:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768128/real 1775768131]  req@ffff9f14852d3800 x1862022114842240/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768153 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768131.595104:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3800 x1862022114842240/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768153 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768131.595111:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768131.595128:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768131.595131:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114842240:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775768131.595137:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768128/real 1775768131]  req@ffff9f14852d2680 x1862022114842112/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768153 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768131.595147:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2680 x1862022114842112/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768153 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768131.595154:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768131.595157:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768131.595160:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114842112:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768131.595166:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768128/real 1775768131]  req@ffff9f14852d2a00 x1862022114841856/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768153 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768131.595175:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2a00 x1862022114841856/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768153 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768131.595182:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768131.595185:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768131.595187:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114841856:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775768131.595194:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768128/real 1775768131]  req@ffff9f14852d1500 x1862022114841984/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768153 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768131.595205:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1500 x1862022114841984/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768153 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768131.595212:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768131.595214:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768131.595217:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114841984:192.168.202.110@tcp:38:kworker.0
00010000:00010000:3.0:1775768131.850697:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775768132.618605:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768132.874749:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775768133.642625:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:2.0:1775768133.642728:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768133.642734:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1775768133.642738:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768133.642747:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5511
00000100:00080000:2.0:1775768133.642771:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768133.642779:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775768133.642816:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0a80 x1862022114842368/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768133.642868:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768133.642871:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775768133.642873:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768133.642877:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5511
00000100:00080000:2.0:1775768133.642881:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768133.642885:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768133.642895:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775768133.642909:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114842368:192.168.202.110@tcp:250:kworker.0
00000100:00100000:2.0:1775768133.642910:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1f80 x1862022114842496/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768133.642925:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768133.642929:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775768133.642931:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768133.642936:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5511
00000100:00080000:2.0:1775768133.642940:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768133.642944:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768133.642949:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775768133.642960:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1500 x1862022114842624/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768133.642970:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768133.642973:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775768133.642974:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768133.642978:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5511
00000800:00000100:0.0:1775768133.642983:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:2.0:1775768133.642984:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768133.642990:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768133.642995:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775768133.643007:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2a00 x1862022114842752/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768133.643015:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768133.643023:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1775768133.643024:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775768133.643029:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768133/real 1775768133]  req@ffff9f14852d0a80 x1862022114842368/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768158 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775768133.643030:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5511
00000100:00080000:2.0:1775768133.643036:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768133.643042:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768133.643049:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775768133.643058:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0a80 x1862022114842368/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768158 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775768133.643063:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2680 x1862022114842880/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775768133.643079:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768133.643084:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768133.643089:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114842368:192.168.202.110@tcp:250:kworker.0
00000100:00080000:2.0:1775768133.643093:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768133.643097:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775768133.643099:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768133.643104:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5511
00000100:00080000:2.0:1775768133.643108:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768133.643112:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775768133.643115:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114842496:192.168.202.110@tcp:38:kworker.0
00000008:00100000:2.0:1775768133.643119:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775768133.643134:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3800 x1862022114843008/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1775768133.643145:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:2.0:1775768133.643145:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768133.643150:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775768133.643152:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768133.643156:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5511
00000100:00080000:2.0:1775768133.643161:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768133.643164:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775768133.643169:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768133/real 1775768133]  req@ffff9f14852d1f80 x1862022114842496/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768158 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:2.0:1775768133.643171:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775768133.643185:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3b80 x1862022114843136/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775768133.643197:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1f80 x1862022114842496/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768158 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775768133.643198:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768133.643206:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775768133.643210:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775768133.643216:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775768133.643218:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5511
00000100:00080000:0.0:1775768133.643223:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00080000:2.0:1775768133.643224:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1775768133.643228:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114842496:192.168.202.110@tcp:38:kworker.0
00000100:00080000:2.0:1775768133.643232:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775768133.643239:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114842624:192.168.202.110@tcp:38:kworker.0
00000008:00100000:2.0:1775768133.643240:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775768133.643256:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0380 x1862022114843264/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1775768133.643265:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768133.643280:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768133/real 1775768133]  req@ffff9f14852d1500 x1862022114842624/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768158 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768133.643294:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1500 x1862022114842624/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768158 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768133.643302:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768133.643305:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768133.643308:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114842624:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768133.643315:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114842752:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768133.643332:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768133.643342:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768133/real 1775768133]  req@ffff9f14852d2a00 x1862022114842752/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768158 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768133.643352:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2a00 x1862022114842752/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768158 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768133.643364:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768133.643367:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768133.643370:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114842752:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775768133.643375:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114842880:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768133.643392:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768133.643401:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768133/real 1775768133]  req@ffff9f14852d2680 x1862022114842880/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768158 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768133.643411:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2680 x1862022114842880/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768158 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768133.643418:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768133.643421:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768133.643424:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114842880:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775768133.643439:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114843008:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768133.643455:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768133.643464:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768133/real 1775768133]  req@ffff9f14852d3800 x1862022114843008/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768158 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768133.643474:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3800 x1862022114843008/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768158 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768133.643480:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768133.643483:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768133.643486:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114843008:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768133.643492:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114843136:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768133.643506:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768133.643521:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768133/real 1775768133]  req@ffff9f14852d3b80 x1862022114843136/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768158 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768133.643531:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3b80 x1862022114843136/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768158 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768133.643539:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768133.643542:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768133.643545:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114843136:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768133.643550:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114843264:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768133.643592:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768133.643603:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768133/real 1775768133]  req@ffff9f14852d0380 x1862022114843264/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768158 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768133.643613:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0380 x1862022114843264/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768158 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768133.643620:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768133.643623:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768133.643626:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114843264:192.168.202.110@tcp:8:kworker.0
00010000:00010000:0.0:1775768133.898650:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775768134.666646:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768134.922728:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775768135.690645:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775768135.946740:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775768136.714650:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775768136.970733:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775768137.738631:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768137.994676:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775768138.762666:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:1.0:1775768138.762666:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775768138.762674:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1775768138.762677:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775768138.762685:0:8:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5516
00000100:00080000:1.0:1775768138.762695:0:8:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775768138.762701:0:8:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775768138.762737:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fd880 x1862022114843392/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775768138.762753:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775768138.762755:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775768138.762756:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775768138.762759:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5516
00000100:00080000:1.0:1775768138.762761:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775768138.762763:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775768138.762771:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775768138.762790:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fe300 x1862022114843520/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775768138.762797:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1775768138.762799:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114843392:192.168.202.110@tcp:250:kworker.0
00000100:00080000:1.0:1775768138.762800:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775768138.762803:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775768138.762808:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5516
00000100:00080000:1.0:1775768138.762810:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775768138.762812:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775768138.762816:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775768138.762825:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdf80 x1862022114843648/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775768138.762830:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775768138.762832:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775768138.762833:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775768138.762836:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5516
00000100:00080000:1.0:1775768138.762845:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775768138.762847:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775768138.762849:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:0.0:1775768138.762853:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775768138.762857:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fea00 x1862022114843776/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775768138.762863:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775768138.762865:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1775768138.762867:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775768138.762870:0:8:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5516
00000100:00080000:1.0:1775768138.762872:0:8:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775768138.762874:0:8:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775768138.762878:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775768138.762885:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdc00 x1862022114843904/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775768138.762888:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768138/real 1775768138]  req@ffff9f14ab4fd880 x1862022114843392/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768163 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775768138.762915:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775768138.762917:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775768138.762919:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775768138.762921:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fd880 x1862022114843392/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768163 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775768138.762923:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5516
00000100:00080000:1.0:1775768138.762927:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775768138.762933:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775768138.762937:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00080000:0.0:1775768138.762939:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00100000:1.0:1775768138.762943:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fce00 x1862022114844032/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775768138.762946:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768138.762955:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114843392:192.168.202.110@tcp:250:kworker.0
00000100:00080000:1.0:1775768138.762955:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775768138.762960:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775768138.762963:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775768138.762967:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5516
00000100:00080000:1.0:1775768138.762969:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775768138.762971:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775768138.762974:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775768138.762979:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff100 x1862022114844160/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775768138.762984:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775768138.762986:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775768138.762986:0:8:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775768138.762989:0:8:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5516
00000100:00080000:1.0:1775768138.762990:0:8:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1775768138.762993:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114843520:192.168.202.110@tcp:38:kworker.0
00000100:00080000:1.0:1775768138.762994:0:8:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775768138.763001:0:8:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775768138.763009:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fc380 x1862022114844288/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1775768138.763027:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768138.763040:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768138/real 1775768138]  req@ffff9f14ab4fe300 x1862022114843520/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768163 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768138.763052:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fe300 x1862022114843520/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768163 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768138.763060:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768138.763063:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768138.763065:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114843520:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768138.763069:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114843648:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768138.763084:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768138.763090:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768138/real 1775768138]  req@ffff9f14ab4fdf80 x1862022114843648/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768163 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768138.763105:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdf80 x1862022114843648/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768163 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768138.763113:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768138.763115:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768138.763118:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114843648:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768138.763124:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114843776:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768138.763142:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768138.763151:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768138/real 1775768138]  req@ffff9f14ab4fea00 x1862022114843776/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768163 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768138.763160:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fea00 x1862022114843776/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768163 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768138.763165:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768138.763167:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768138.763169:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114843776:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775768138.763173:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114843904:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768138.763188:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768138.763203:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768138/real 1775768138]  req@ffff9f14ab4fdc00 x1862022114843904/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768163 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768138.763220:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdc00 x1862022114843904/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768163 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768138.763226:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768138.763228:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768138.763230:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114843904:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775768138.763234:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114844032:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768138.763252:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768138.763258:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768138/real 1775768138]  req@ffff9f14ab4fce00 x1862022114844032/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768163 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768138.763265:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fce00 x1862022114844032/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768163 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768138.763269:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768138.763271:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768138.763273:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114844032:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768138.763276:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114844160:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768138.763284:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768138.763288:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768138/real 1775768138]  req@ffff9f14ab4ff100 x1862022114844160/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768163 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768138.763294:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff100 x1862022114844160/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768163 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768138.763302:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768138.763304:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768138.763306:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114844160:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768138.763317:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114844288:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768138.763325:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768138.763330:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768138/real 1775768138]  req@ffff9f14ab4fc380 x1862022114844288/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768163 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768138.763336:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fc380 x1862022114844288/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768163 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768138.763350:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768138.763352:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768138.763353:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114844288:192.168.202.110@tcp:8:kworker.0
00010000:00010000:3.0:1775768139.018706:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775768139.786659:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768140.042682:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775768140.810641:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:0.0:1775768141.066701:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775768141.834655:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768142.090728:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775768142.858638:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775768143.114680:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775768143.882643:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775768143.882657:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775768143.882663:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775768143.882668:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768143.882678:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5521
00000100:00080000:3.0:1775768143.882687:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768143.882691:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775768143.882721:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744700 x1862022114844416/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775768143.882762:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775768143.882764:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775768143.882766:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768143.882768:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5521
00000100:00080000:3.0:1775768143.882771:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768143.882773:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775768143.882782:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775768143.882792:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745f80 x1862022114844544/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775768143.882800:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1775768143.882802:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114844416:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775768143.882803:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775768143.882805:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768143.882812:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5521
00000100:00080000:3.0:1775768143.882813:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768143.882815:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775768143.882817:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775768143.882823:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747800 x1862022114844672/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775768143.882828:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775768143.882831:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775768143.882832:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768143.882834:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5521
00000100:00080000:3.0:1775768143.882836:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768143.882838:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775768143.882841:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775768143.882848:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744000 x1862022114844800/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1775768143.882857:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775768143.882857:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775768143.882862:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775768143.882864:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768143.882868:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5521
00000100:00080000:3.0:1775768143.882873:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768143.882877:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775768143.882881:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775768143.882890:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768143/real 1775768143]  req@ffff9f1485744700 x1862022114844416/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768168 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775768143.882894:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747100 x1862022114844928/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775768143.882917:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775768143.882921:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775768143.882922:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768143.882926:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5521
00000100:00080000:3.0:1775768143.882929:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1775768143.882932:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744700 x1862022114844416/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768168 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775768143.882934:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775768143.882941:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00080000:0.0:1775768143.882950:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00100000:3.0:1775768143.882952:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745180 x1862022114845056/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775768143.882958:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:3.0:1775768143.882961:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775768143.882964:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00100000:0.0:1775768143.882965:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114844416:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775768143.882966:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768143.882973:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5521
00000100:00080000:3.0:1775768143.882975:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768143.882978:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775768143.882982:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775768143.882991:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745c00 x1862022114845184/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775768143.882993:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114844544:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775768143.883004:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775768143.883008:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775768143.883009:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768143.883014:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5521
00000100:00080000:3.0:1775768143.883018:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000800:00000100:0.0:1775768143.883023:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775768143.883024:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775768143.883029:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775768143.883040:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768143/real 1775768143]  req@ffff9f1485745f80 x1862022114844544/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768168 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775768143.883044:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746680 x1862022114845312/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775768143.883060:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745f80 x1862022114844544/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768168 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768143.883068:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768143.883072:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768143.883076:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114844544:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768143.883082:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114844672:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768143.883101:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768143.883110:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768143/real 1775768143]  req@ffff9f1485747800 x1862022114844672/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768168 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768143.883117:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747800 x1862022114844672/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768168 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768143.883126:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768143.883127:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768143.883129:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114844672:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768143.883132:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114844800:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768143.883144:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768143.883149:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768143/real 1775768143]  req@ffff9f1485744000 x1862022114844800/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768168 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768143.883164:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744000 x1862022114844800/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768168 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768143.883168:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768143.883170:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768143.883172:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114844800:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775768143.883175:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114844928:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768143.883186:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768143.883195:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768143/real 1775768143]  req@ffff9f1485747100 x1862022114844928/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768168 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768143.883213:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747100 x1862022114844928/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768168 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768143.883221:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768143.883224:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768143.883228:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114844928:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775768143.883234:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114845056:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768143.883247:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768143.883253:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768143/real 1775768143]  req@ffff9f1485745180 x1862022114845056/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768168 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768143.883260:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745180 x1862022114845056/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768168 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768143.883264:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768143.883266:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768143.883268:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114845056:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768143.883279:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114845184:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768143.883287:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768143.883292:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768143/real 1775768143]  req@ffff9f1485745c00 x1862022114845184/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768168 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768143.883298:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745c00 x1862022114845184/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768168 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768143.883302:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768143.883303:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768143.883305:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114845184:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768143.883308:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114845312:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768143.883315:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768143.883320:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768143/real 1775768143]  req@ffff9f1485746680 x1862022114845312/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768168 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768143.883326:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746680 x1862022114845312/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768168 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768143.883330:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768143.883332:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768143.883333:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114845312:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775768144.138795:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775768144.906640:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768145.162675:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775768145.930679:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768146.186760:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775768146.954666:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768147.210798:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775768147.978730:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:0.0:1775768148.234775:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775768149.002678:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:2.0:1775768149.002708:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768149.002723:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1775768149.002727:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768149.002735:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5526
00000100:00080000:2.0:1775768149.002745:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768149.002749:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775768149.002781:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0700 x1862022114845440/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768149.002808:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768149.002811:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775768149.002812:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768149.002815:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5526
00000100:00080000:2.0:1775768149.002817:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768149.002819:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768149.002826:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775768149.002833:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2d80 x1862022114845568/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768149.002841:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768149.002844:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775768149.002845:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768149.002849:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5526
00000100:00080000:2.0:1775768149.002853:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768149.002856:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768149.002861:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775768149.002874:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114845440:192.168.202.110@tcp:250:kworker.0
00000100:00100000:2.0:1775768149.002874:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0000 x1862022114845696/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768149.002899:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768149.002903:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775768149.002904:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768149.002908:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5526
00000100:00080000:2.0:1775768149.002911:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768149.002914:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768149.002918:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775768149.002928:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1180 x1862022114845824/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768149.002933:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768149.002935:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1775768149.002937:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768149.002939:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5526
00000100:00080000:2.0:1775768149.002942:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768149.002944:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768149.002947:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:0.0:1775768149.002952:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775768149.002956:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1880 x1862022114845952/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768149.002963:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768149.002966:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775768149.002967:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768149.002973:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5526
00000100:00080000:2.0:1775768149.002977:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768149.002980:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768149.002984:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775768149.002994:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0e00 x1862022114846080/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768149.003000:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768149.003002:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775768149.003004:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768149.003010:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5526
00000100:00100000:0.0:1775768149.003011:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768149/real 1775768149]  req@ffff9f14852d0700 x1862022114845440/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768174 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775768149.003012:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768149.003022:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768149.003032:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775768149.003050:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2300 x1862022114846208/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775768149.003058:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0700 x1862022114845440/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768174 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775768149.003077:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768149.003087:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775768149.003089:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768149.003093:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5526
00000100:00080000:0.0:1775768149.003095:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775768149.003096:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775768149.003108:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:2.0:1775768149.003108:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775768149.003120:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114845440:192.168.202.110@tcp:250:kworker.0
00000008:00100000:2.0:1775768149.003120:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775768149.003133:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114845568:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775768149.003143:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3100 x1862022114846336/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1775768149.003161:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768149.003175:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768149/real 1775768149]  req@ffff9f14852d2d80 x1862022114845568/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768174 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768149.003192:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2d80 x1862022114845568/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768174 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768149.003197:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768149.003200:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768149.003202:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114845568:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768149.003224:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114845696:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768149.003235:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768149.003242:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768149/real 1775768149]  req@ffff9f14852d0000 x1862022114845696/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768174 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768149.003249:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0000 x1862022114845696/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768174 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768149.003253:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768149.003255:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768149.003256:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114845696:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768149.003260:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114845824:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768149.003271:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768149.003278:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768149/real 1775768149]  req@ffff9f14852d1180 x1862022114845824/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768174 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768149.003285:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1180 x1862022114845824/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768174 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768149.003294:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768149.003296:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768149.003298:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114845824:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775768149.003302:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114845952:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768149.003313:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768149.003318:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768149/real 1775768149]  req@ffff9f14852d1880 x1862022114845952/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768174 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768149.003324:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1880 x1862022114845952/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768174 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768149.003328:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768149.003330:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768149.003332:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114845952:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775768149.003335:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114846080:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768149.003344:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768149.003354:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768149/real 1775768149]  req@ffff9f14852d0e00 x1862022114846080/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768174 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768149.003364:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0e00 x1862022114846080/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768174 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768149.003371:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768149.003374:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768149.003377:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114846080:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768149.003383:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114846208:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768149.003403:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768149.003413:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768149/real 1775768149]  req@ffff9f14852d2300 x1862022114846208/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768174 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768149.003424:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2300 x1862022114846208/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768174 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768149.003432:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768149.003435:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768149.003437:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114846208:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768149.003441:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114846336:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768149.003456:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768149.003462:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768149/real 1775768149]  req@ffff9f14852d3100 x1862022114846336/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768174 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768149.003469:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3100 x1862022114846336/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768174 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768149.003474:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768149.003476:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768149.003478:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114846336:192.168.202.110@tcp:8:kworker.0
00010000:00010000:3.0:1775768149.258764:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775768150.026753:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775768150.282752:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775768151.050686:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775768151.306795:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775768152.074688:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768152.330729:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775768153.098638:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768153.354711:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768154.122652:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775768154.122652:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775768154.122662:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775768154.122665:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768154.122676:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5531
00000100:00080000:3.0:1775768154.122687:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768154.122694:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775768154.122728:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745500 x1862022114846464/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775768154.122753:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775768154.122757:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775768154.122758:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768154.122762:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5531
00000100:00080000:3.0:1775768154.122766:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768154.122770:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775768154.122779:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775768154.122784:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114846464:192.168.202.110@tcp:250:kworker.0
00000100:00100000:3.0:1775768154.122794:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744e00 x1862022114846592/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775768154.122805:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775768154.122809:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775768154.122810:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768154.122814:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5531
00000100:00080000:3.0:1775768154.122818:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768154.122822:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775768154.122827:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775768154.122838:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744380 x1862022114846720/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1775768154.122843:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775768154.122857:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775768154.122860:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775768154.122862:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768154.122866:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5531
00000100:00080000:3.0:1775768154.122870:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1775768154.122876:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768154/real 1775768154]  req@ffff9f1485745500 x1862022114846464/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768179 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775768154.122876:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775768154.122886:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775768154.122893:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745500 x1862022114846464/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768179 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775768154.122900:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746a00 x1862022114846848/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775768154.122901:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768154.122908:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768154.122914:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114846464:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775768154.122919:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775768154.122923:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775768154.122925:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768154.122929:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5531
00000100:00080000:3.0:1775768154.122933:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1775768154.122934:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114846592:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775768154.122940:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775768154.122946:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775768154.122959:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744a80 x1862022114846976/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1775768154.122960:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768154.122973:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768154/real 1775768154]  req@ffff9f1485744e00 x1862022114846592/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768179 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775768154.122973:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775768154.122981:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775768154.122985:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768154.122992:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5531
00000100:00080000:3.0:1775768154.122997:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1775768154.122998:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744e00 x1862022114846592/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768179 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775768154.123003:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775768154.123010:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00080000:0.0:1775768154.123013:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768154.123018:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768154.123021:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114846592:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775768154.123025:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747480 x1862022114847104/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775768154.123029:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114846720:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775768154.123043:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775768154.123046:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775768154.123049:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000800:00000100:0.0:1775768154.123052:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775768154.123058:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5531
00000100:00100000:0.0:1775768154.123063:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768154/real 1775768154]  req@ffff9f1485744380 x1862022114846720/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768179 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775768154.123063:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768154.123072:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775768154.123077:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744380 x1862022114846720/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768179 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:3.0:1775768154.123080:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00080000:0.0:1775768154.123083:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768154.123086:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768154.123089:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114846720:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768154.123093:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114846848:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775768154.123097:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745880 x1862022114847232/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1775768154.123107:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775768154.123109:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775768154.123113:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00100000:0.0:1775768154.123114:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768154/real 1775768154]  req@ffff9f1485746a00 x1862022114846848/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768179 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775768154.123116:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768154.123127:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5531
00000100:00100000:0.0:1775768154.123129:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746a00 x1862022114846848/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768179 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775768154.123134:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775768154.123137:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775768154.123141:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00080000:0.0:1775768154.123142:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768154.123147:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114846848:192.168.202.110@tcp:8:kworker.0
00000008:00100000:3.0:1775768154.123150:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775768154.123158:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114846976:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775768154.123166:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746300 x1862022114847360/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1775768154.123172:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768154.123179:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768154/real 1775768154]  req@ffff9f1485744a80 x1862022114846976/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768179 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768154.123210:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744a80 x1862022114846976/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768179 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768154.123220:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768154.123223:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768154.123227:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114846976:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775768154.123232:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114847104:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768154.123251:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768154.123262:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768154/real 1775768154]  req@ffff9f1485747480 x1862022114847104/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768179 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768154.123272:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747480 x1862022114847104/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768179 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768154.123280:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768154.123283:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768154.123286:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114847104:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768154.123292:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114847232:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768154.123309:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768154.123318:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768154/real 1775768154]  req@ffff9f1485745880 x1862022114847232/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768179 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768154.123330:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745880 x1862022114847232/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768179 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768154.123337:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768154.123340:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768154.123344:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114847232:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768154.123358:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114847360:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768154.123375:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768154.123384:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768154/real 1775768154]  req@ffff9f1485746300 x1862022114847360/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768179 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768154.123395:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746300 x1862022114847360/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768179 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768154.123403:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768154.123405:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768154.123408:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114847360:192.168.202.110@tcp:8:kworker.0
00010000:00010000:3.0:1775768154.378715:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768155.146685:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775768155.402709:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768156.170863:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768156.426780:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775768157.194706:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775768157.450699:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775768158.218684:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775768158.474734:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775768159.242666:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:2.0:1775768159.242708:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768159.242718:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1775768159.242722:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768159.242733:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5536
00000100:00080000:2.0:1775768159.242746:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768159.242752:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775768159.242790:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3480 x1862022114847488/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768159.242819:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768159.242822:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775768159.242823:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768159.242827:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5536
00000100:00080000:2.0:1775768159.242829:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768159.242832:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768159.242840:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775768159.242848:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1c00 x1862022114847616/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768159.242855:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768159.242858:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775768159.242859:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768159.242862:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5536
00000100:00080000:2.0:1775768159.242864:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1775768159.242868:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114847488:192.168.202.110@tcp:250:kworker.0
00000100:00080000:2.0:1775768159.242868:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768159.242874:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775768159.242884:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3100 x1862022114847744/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768159.242891:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768159.242893:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775768159.242894:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768159.242897:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5536
00000100:00080000:2.0:1775768159.242898:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768159.242900:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768159.242904:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775768159.242912:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2300 x1862022114847872/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768159.242918:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768159.242920:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1775768159.242921:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768159.242923:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5536
00000100:00080000:2.0:1775768159.242925:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768159.242926:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768159.242929:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775768159.242935:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0e00 x1862022114848000/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768159.242940:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000800:00000100:0.0:1775768159.242942:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:2.0:1775768159.242943:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775768159.242946:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768159.242950:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5536
00000100:00080000:2.0:1775768159.242952:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768159.242956:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768159.242959:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775768159.242965:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1880 x1862022114848128/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768159.242970:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768159.242972:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775768159.242974:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768159.242977:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5536
00000100:00080000:2.0:1775768159.242979:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768159.242982:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768159.242985:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775768159.242987:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768159/real 1775768159]  req@ffff9f14852d3480 x1862022114847488/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768184 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775768159.242994:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1180 x1862022114848256/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768159.243002:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768159.243004:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775768159.243005:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768159.243008:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5536
00000100:00080000:2.0:1775768159.243010:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1775768159.243013:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3480 x1862022114847488/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768184 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775768159.243014:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768159.243018:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775768159.243027:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0000 x1862022114848384/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775768159.243031:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768159.243038:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768159.243047:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114847488:192.168.202.110@tcp:250:kworker.0
00000100:00100000:0.0:1775768159.243055:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114847616:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768159.243081:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768159.243096:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768159/real 1775768159]  req@ffff9f14852d1c00 x1862022114847616/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768184 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768159.243108:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1c00 x1862022114847616/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768184 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768159.243117:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768159.243120:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768159.243123:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114847616:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768159.243148:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114847744:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768159.243162:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768159.243170:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768159/real 1775768159]  req@ffff9f14852d3100 x1862022114847744/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768184 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768159.243181:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3100 x1862022114847744/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768184 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768159.243189:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768159.243192:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768159.243195:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114847744:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768159.243201:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114847872:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768159.243221:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768159.243231:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768159/real 1775768159]  req@ffff9f14852d2300 x1862022114847872/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768184 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768159.243241:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2300 x1862022114847872/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768184 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768159.243247:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768159.243249:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768159.243251:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114847872:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775768159.243255:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114848000:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768159.243268:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768159.243273:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768159/real 1775768159]  req@ffff9f14852d0e00 x1862022114848000/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768184 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768159.243280:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0e00 x1862022114848000/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768184 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768159.243284:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768159.243286:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768159.243287:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114848000:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775768159.243290:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114848128:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768159.243300:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768159.243304:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768159/real 1775768159]  req@ffff9f14852d1880 x1862022114848128/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768184 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768159.243311:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1880 x1862022114848128/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768184 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768159.243315:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768159.243316:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768159.243319:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114848128:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768159.243321:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114848256:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768159.243329:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768159.243333:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768159/real 1775768159]  req@ffff9f14852d1180 x1862022114848256/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768184 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768159.243339:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1180 x1862022114848256/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768184 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768159.243343:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768159.243344:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768159.243346:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114848256:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768159.243348:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114848384:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768159.243356:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768159.243360:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768159/real 1775768159]  req@ffff9f14852d0000 x1862022114848384/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768184 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768159.243367:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0000 x1862022114848384/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768184 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768159.243371:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768159.243372:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768159.243374:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114848384:192.168.202.110@tcp:8:kworker.0
00010000:00010000:3.0:1775768159.498754:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768160.266718:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768160.522705:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768161.290712:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768161.546729:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775768162.314736:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775768162.570762:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775768163.338663:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768163.594726:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1775768164.362690:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768164.362706:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1775768164.362712:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768164.362727:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5541
00000100:00080000:2.0:1775768164.362743:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768164.362753:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775768164.362757:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:2.0:1775768164.362802:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2d80 x1862022114848512/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768164.362839:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768164.362844:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775768164.362847:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768164.362864:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5541
00000100:00080000:2.0:1775768164.362870:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1775768164.362880:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114848512:192.168.202.110@tcp:250:kworker.0
00000100:00080000:2.0:1775768164.362880:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768164.362903:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775768164.362926:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0700 x1862022114848640/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768164.362941:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768164.362947:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775768164.362949:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768164.362955:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5541
00000100:00080000:2.0:1775768164.362960:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768164.362965:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768164.362973:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000800:00000100:0.0:1775768164.362980:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775768164.362991:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0380 x1862022114848768/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768164.363006:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768164.363011:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775768164.363013:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768164.363020:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5541
00000100:00080000:2.0:1775768164.363026:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768164.363032:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775768164.363038:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768164/real 1775768164]  req@ffff9f14852d2d80 x1862022114848512/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768189 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:2.0:1775768164.363043:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775768164.363065:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3b80 x1862022114848896/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775768164.363074:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2d80 x1862022114848512/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768189 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775768164.363096:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775768164.363101:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775768164.363101:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1775768164.363109:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775768164.363117:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:2.0:1775768164.363123:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5541
00000100:00100000:0.0:1775768164.363124:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114848512:192.168.202.110@tcp:250:kworker.0
00000100:00080000:2.0:1775768164.363132:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768164.363139:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768164.363149:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775768164.363166:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3800 x1862022114849024/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775768164.363179:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114848640:192.168.202.110@tcp:38:kworker.0
00000100:00080000:2.0:1775768164.363180:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768164.363195:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775768164.363198:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768164.363206:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5541
00000100:00080000:2.0:1775768164.363211:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768164.363217:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000800:00000100:0.0:1775768164.363224:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000008:00100000:2.0:1775768164.363224:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775768164.363246:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768164/real 1775768164]  req@ffff9f14852d0700 x1862022114848640/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768189 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775768164.363247:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2680 x1862022114849152/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768164.363297:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1775768164.363304:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0700 x1862022114848640/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768189 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775768164.363305:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775768164.363311:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768164.363333:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5541
00000100:00080000:0.0:1775768164.363336:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775768164.363341:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775768164.363345:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00080000:2.0:1775768164.363351:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775768164.363358:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114848640:192.168.202.110@tcp:38:kworker.0
00000008:00100000:2.0:1775768164.363363:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775768164.363371:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114848768:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775768164.363381:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2a00 x1862022114849280/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768164.363395:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768164.363400:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000800:00000100:0.0:1775768164.363406:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:2.0:1775768164.363406:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768164.363417:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5541
00000100:00080000:2.0:1775768164.363424:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1775768164.363427:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768164/real 1775768164]  req@ffff9f14852d0380 x1862022114848768/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768189 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775768164.363433:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768164.363446:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775768164.363471:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1500 x1862022114849408/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775768164.363472:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0380 x1862022114848768/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768189 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768164.363527:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768164.363550:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768164.363621:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114848768:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768164.363631:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114848896:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768164.363662:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768164.363674:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768164/real 1775768164]  req@ffff9f14852d3b80 x1862022114848896/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768189 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768164.363686:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3b80 x1862022114848896/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768189 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768164.363694:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768164.363697:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768164.363700:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114848896:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775768164.363716:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114849024:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768164.363733:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768164.363742:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768164/real 1775768164]  req@ffff9f14852d3800 x1862022114849024/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768189 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768164.363753:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3800 x1862022114849024/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768189 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768164.363780:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768164.363783:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768164.363786:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114849024:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775768164.363792:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114849152:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768164.363811:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768164.363821:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768164/real 1775768164]  req@ffff9f14852d2680 x1862022114849152/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768189 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768164.363831:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2680 x1862022114849152/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768189 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768164.363838:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768164.363842:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768164.363844:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114849152:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768164.363849:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114849280:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768164.363864:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768164.363872:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768164/real 1775768164]  req@ffff9f14852d2a00 x1862022114849280/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768189 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768164.363882:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2a00 x1862022114849280/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768189 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768164.363888:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768164.363891:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768164.363895:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114849280:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768164.363901:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114849408:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768164.363915:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768164.363923:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768164/real 1775768164]  req@ffff9f14852d1500 x1862022114849408/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768189 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768164.363934:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1500 x1862022114849408/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768189 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768164.363942:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768164.363945:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768164.363948:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114849408:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775768164.618775:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775768165.386723:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775768165.642780:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775768166.410689:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768166.666717:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775768167.434695:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768167.690782:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775768168.458660:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768168.714724:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1775768169.482691:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1775768169.482694:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:2.0:1775768169.482704:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1775768169.482709:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768169.482721:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5546
00000100:00080000:2.0:1775768169.482736:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768169.482744:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775768169.482788:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1f80 x1862022114849536/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768169.482822:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768169.482826:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775768169.482828:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768169.482834:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5546
00000100:00080000:2.0:1775768169.482838:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768169.482843:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775768169.482847:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114849536:192.168.202.110@tcp:250:kworker.0
00000008:00100000:2.0:1775768169.482854:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775768169.482871:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0a80 x1862022114849664/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768169.482883:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768169.482886:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775768169.482889:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768169.482893:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5546
00000100:00080000:2.0:1775768169.482897:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768169.482901:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768169.482907:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000800:00000100:0.0:1775768169.482913:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775768169.482922:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1500 x1862022114849792/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768169.482937:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768169.482941:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775768169.482943:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768169.482948:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5546
00000100:00100000:0.0:1775768169.482955:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768169/real 1775768169]  req@ffff9f14852d1f80 x1862022114849536/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768194 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775768169.482962:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1775768169.482971:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1f80 x1862022114849536/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768194 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775768169.482971:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768169.482979:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00080000:0.0:1775768169.482989:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768169.482997:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775768169.482998:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2a00 x1862022114849920/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775768169.483006:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114849536:192.168.202.110@tcp:250:kworker.0
00000100:00080000:2.0:1775768169.483015:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768169.483019:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1775768169.483021:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768169.483026:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5546
00000100:00080000:2.0:1775768169.483030:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768169.483034:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775768169.483038:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114849664:192.168.202.110@tcp:38:kworker.0
00000008:00100000:2.0:1775768169.483040:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775768169.483056:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2680 x1862022114850048/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1775768169.483064:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:2.0:1775768169.483072:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768169.483076:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00100000:0.0:1775768169.483078:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768169/real 1775768169]  req@ffff9f14852d0a80 x1862022114849664/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768194 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775768169.483079:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768169.483087:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5546
00000100:00080000:2.0:1775768169.483095:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1775768169.483102:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0a80 x1862022114849664/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768194 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775768169.483102:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768169.483110:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00080000:0.0:1775768169.483116:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768169.483122:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775768169.483126:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3800 x1862022114850176/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775768169.483130:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114849664:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768169.483143:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114849792:192.168.202.110@tcp:38:kworker.0
00000100:00080000:2.0:1775768169.483147:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768169.483151:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775768169.483153:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768169.483158:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5546
00000100:00080000:2.0:1775768169.483163:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000800:00000100:0.0:1775768169.483170:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:2.0:1775768169.483177:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775768169.483183:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768169/real 1775768169]  req@ffff9f14852d1500 x1862022114849792/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768194 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:2.0:1775768169.483187:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775768169.483201:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1500 x1862022114849792/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768194 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775768169.483206:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3b80 x1862022114850304/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775768169.483215:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768169.483222:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00080000:2.0:1775768169.483225:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1775768169.483228:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114849792:192.168.202.110@tcp:38:kworker.0
00000100:00080000:2.0:1775768169.483232:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775768169.483236:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775768169.483239:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114849920:192.168.202.110@tcp:8:kworker.0
00000100:00080000:2.0:1775768169.483243:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5546
00000100:00080000:2.0:1775768169.483250:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768169.483255:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768169.483261:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:0.0:1775768169.483269:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775768169.483276:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0380 x1862022114850432/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775768169.483281:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768169/real 1775768169]  req@ffff9f14852d2a00 x1862022114849920/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768194 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768169.483302:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2a00 x1862022114849920/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768194 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768169.483313:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768169.483316:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768169.483320:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114849920:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775768169.483337:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114850048:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768169.483357:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768169.483367:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768169/real 1775768169]  req@ffff9f14852d2680 x1862022114850048/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768194 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768169.483378:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2680 x1862022114850048/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768194 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768169.483386:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768169.483388:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768169.483391:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114850048:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775768169.483397:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114850176:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768169.483412:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768169.483418:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768169/real 1775768169]  req@ffff9f14852d3800 x1862022114850176/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768194 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768169.483425:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3800 x1862022114850176/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768194 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768169.483430:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768169.483432:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768169.483434:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114850176:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768169.483437:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114850304:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768169.483445:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768169.483449:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768169/real 1775768169]  req@ffff9f14852d3b80 x1862022114850304/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768194 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768169.483456:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3b80 x1862022114850304/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768194 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768169.483460:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768169.483462:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768169.483463:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114850304:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768169.483466:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114850432:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768169.483475:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768169.483480:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768169/real 1775768169]  req@ffff9f14852d0380 x1862022114850432/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768194 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768169.483508:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0380 x1862022114850432/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768194 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768169.483515:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768169.483518:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768169.483522:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114850432:192.168.202.110@tcp:8:kworker.0
00010000:00010000:1.0:1775768169.738753:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775768170.506674:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768170.762706:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775768171.530672:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775768171.786729:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775768172.554652:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775768172.810694:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775768173.578626:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768173.834714:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768174.602670:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775768174.602672:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775768174.602680:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775768174.602685:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768174.602694:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5552
00000100:00080000:3.0:1775768174.602705:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768174.602713:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775768174.602752:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747b80 x1862022114850560/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775768174.602778:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775768174.602782:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775768174.602784:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768174.602788:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5552
00000100:00080000:3.0:1775768174.602791:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768174.602796:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775768174.602805:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114850560:192.168.202.110@tcp:250:kworker.0
00000008:00100000:3.0:1775768174.602806:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775768174.602825:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746300 x1862022114850688/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775768174.602836:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775768174.602840:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775768174.602841:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768174.602845:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5552
00000100:00080000:3.0:1775768174.602849:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768174.602854:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000800:00000100:0.0:1775768174.602856:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000008:00100000:3.0:1775768174.602862:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775768174.602875:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745880 x1862022114850816/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775768174.602884:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775768174.602887:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775768174.602889:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768174.602893:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5552
00000100:00080000:3.0:1775768174.602898:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1775768174.602902:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768174/real 1775768174]  req@ffff9f1485747b80 x1862022114850560/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768199 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775768174.602903:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775768174.602910:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775768174.602926:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747480 x1862022114850944/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775768174.602927:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747b80 x1862022114850560/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768199 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768174.602954:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775768174.602956:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775768174.602958:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:3.0:1775768174.602963:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775768174.602966:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768174.602972:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5552
00000100:00100000:0.0:1775768174.602973:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114850560:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775768174.602976:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768174.602982:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775768174.602988:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775768174.602998:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744a80 x1862022114851072/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775768174.603005:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114850688:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775768174.603011:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775768174.603015:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775768174.603017:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768174.603021:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5552
00000100:00080000:3.0:1775768174.603026:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768174.603030:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000800:00000100:0.0:1775768174.603033:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000008:00100000:3.0:1775768174.603039:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775768174.603049:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768174/real 1775768174]  req@ffff9f1485746300 x1862022114850688/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768199 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775768174.603053:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746a00 x1862022114851200/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775768174.603074:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746300 x1862022114850688/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768199 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775768174.603081:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775768174.603088:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775768174.603088:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775768174.603090:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775768174.603096:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00080000:3.0:1775768174.603098:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5552
00000100:00100000:0.0:1775768174.603102:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114850688:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775768174.603104:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768174.603112:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775768174.603119:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775768174.603122:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114850816:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775768174.603133:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744380 x1862022114851328/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775768174.603142:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000800:00000100:0.0:1775768174.603146:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775768174.603149:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775768174.603151:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768174.603157:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5552
00000100:00100000:0.0:1775768174.603161:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768174/real 1775768174]  req@ffff9f1485745880 x1862022114850816/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768199 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775768174.603161:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768174.603168:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775768174.603176:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775768174.603181:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745880 x1862022114850816/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768199 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775768174.603189:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744e00 x1862022114851456/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775768174.603191:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768174.603199:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768174.603206:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114850816:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768174.603213:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114850944:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768174.603235:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768174.603245:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768174/real 1775768174]  req@ffff9f1485747480 x1862022114850944/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768199 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768174.603256:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747480 x1862022114850944/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768199 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768174.603264:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768174.603267:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768174.603270:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114850944:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775768174.603284:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114851072:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768174.603302:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768174.603311:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768174/real 1775768174]  req@ffff9f1485744a80 x1862022114851072/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768199 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768174.603322:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744a80 x1862022114851072/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768199 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768174.603328:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768174.603331:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768174.603334:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114851072:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775768174.603340:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114851200:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768174.603356:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768174.603365:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768174/real 1775768174]  req@ffff9f1485746a00 x1862022114851200/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768199 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768174.603376:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746a00 x1862022114851200/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768199 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768174.603383:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768174.603386:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768174.603390:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114851200:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768174.603395:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114851328:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768174.603409:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768174.603418:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768174/real 1775768174]  req@ffff9f1485744380 x1862022114851328/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768199 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768174.603428:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744380 x1862022114851328/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768199 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768174.603468:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768174.603471:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768174.603475:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114851328:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768174.603481:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114851456:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768174.603499:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768174.603509:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768174/real 1775768174]  req@ffff9f1485744e00 x1862022114851456/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768199 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768174.603520:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744e00 x1862022114851456/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768199 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768174.603527:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768174.603530:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768174.603533:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114851456:192.168.202.110@tcp:8:kworker.0
00010000:00010000:3.0:1775768174.858674:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768175.626627:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768175.882706:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768176.650736:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775768176.906739:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768177.674638:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768177.930699:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768178.698669:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775768178.954754:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1775768179.722664:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768179.722680:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1775768179.722686:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00100000:1.0:1775768179.722693:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:2.0:1775768179.722700:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5557
00000100:00080000:2.0:1775768179.722718:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768179.722728:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775768179.722776:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0700 x1862022114851584/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768179.722812:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768179.722816:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775768179.722819:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768179.722824:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5557
00000100:00080000:2.0:1775768179.722828:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768179.722833:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768179.722845:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775768179.722861:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2d80 x1862022114851712/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775768179.722873:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114851584:192.168.202.110@tcp:250:kworker.0
00000100:00080000:2.0:1775768179.722892:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768179.722896:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775768179.722898:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768179.722904:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5557
00000100:00080000:2.0:1775768179.722909:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768179.722913:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768179.722919:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775768179.722934:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0000 x1862022114851840/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768179.722945:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768179.722949:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775768179.722951:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000800:00000100:0.0:1775768179.722957:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:2.0:1775768179.722961:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5557
00000100:00080000:2.0:1775768179.722968:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768179.722973:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768179.722980:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775768179.722994:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1180 x1862022114851968/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775768179.723003:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768179/real 1775768179]  req@ffff9f14852d0700 x1862022114851584/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768204 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775768179.723009:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768179.723026:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00100000:0.0:1775768179.723028:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0700 x1862022114851584/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768204 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775768179.723030:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768179.723044:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5557
00000100:00080000:0.0:1775768179.723045:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775768179.723052:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768179.723057:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00080000:0.0:1775768179.723066:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000008:00100000:2.0:1775768179.723066:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775768179.723074:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114851584:192.168.202.110@tcp:250:kworker.0
00000100:00100000:2.0:1775768179.723084:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1880 x1862022114852096/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768179.723095:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768179.723100:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775768179.723102:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775768179.723104:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114851712:192.168.202.110@tcp:38:kworker.0
00000100:00080000:2.0:1775768179.723111:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5557
00000100:00080000:2.0:1775768179.723117:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768179.723123:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768179.723129:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000800:00000100:0.0:1775768179.723133:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768179.723147:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768179/real 1775768179]  req@ffff9f14852d2d80 x1862022114851712/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768204 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775768179.723151:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0e00 x1862022114852224/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775768179.723167:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2d80 x1862022114851712/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768204 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775768179.723171:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768179.723184:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:0.0:1775768179.723185:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775768179.723187:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775768179.723190:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00080000:2.0:1775768179.723196:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5557
00000100:00100000:0.0:1775768179.723197:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114851712:192.168.202.110@tcp:38:kworker.0
00000100:00080000:2.0:1775768179.723205:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1775768179.723206:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114851840:192.168.202.110@tcp:38:kworker.0
00000100:00080000:2.0:1775768179.723215:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768179.723222:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000800:00000100:0.0:1775768179.723227:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768179.723236:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768179/real 1775768179]  req@ffff9f14852d0000 x1862022114851840/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768204 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775768179.723244:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2300 x1862022114852352/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775768179.723249:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0000 x1862022114851840/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768204 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768179.723273:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768179.723282:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00080000:2.0:1775768179.723285:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1775768179.723289:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114851840:192.168.202.110@tcp:38:kworker.0
00000100:00080000:2.0:1775768179.723291:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775768179.723295:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775768179.723303:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114851968:192.168.202.110@tcp:8:kworker.0
00000100:00080000:2.0:1775768179.723304:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5557
00000100:00080000:2.0:1775768179.723311:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768179.723316:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768179.723323:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:0.0:1775768179.723334:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775768179.723340:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3100 x1862022114852480/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775768179.723346:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768179/real 1775768179]  req@ffff9f14852d1180 x1862022114851968/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768204 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768179.723368:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1180 x1862022114851968/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768204 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768179.723378:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768179.723395:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768179.723401:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114851968:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775768179.723409:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114852096:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768179.723431:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768179.723440:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768179/real 1775768179]  req@ffff9f14852d1880 x1862022114852096/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768204 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768179.723448:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1880 x1862022114852096/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768204 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768179.723455:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768179.723459:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768179.723462:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114852096:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775768179.723482:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114852224:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768179.723504:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768179.723516:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768179/real 1775768179]  req@ffff9f14852d0e00 x1862022114852224/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768204 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768179.723529:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0e00 x1862022114852224/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768204 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768179.723545:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768179.723549:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768179.723552:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114852224:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768179.723587:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114852352:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768179.723605:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768179.723613:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768179/real 1775768179]  req@ffff9f14852d2300 x1862022114852352/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768204 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768179.723621:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2300 x1862022114852352/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768204 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768179.723626:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768179.723629:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768179.723631:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114852352:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768179.723634:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114852480:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768179.723644:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768179.723650:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768179/real 1775768179]  req@ffff9f14852d3100 x1862022114852480/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768204 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768179.723657:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3100 x1862022114852480/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768204 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768179.723661:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768179.723663:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768179.723665:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114852480:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775768179.978723:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775768180.746673:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768181.002775:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775768181.770686:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768182.026716:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775768182.794653:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768183.050735:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775768183.818684:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775768184.074714:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1775768184.842656:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:1.0:1775768184.842660:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:2.0:1775768184.842666:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1775768184.842671:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768184.842679:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5562
00000100:00080000:2.0:1775768184.842689:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768184.842695:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775768184.842731:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1c00 x1862022114852608/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768184.842755:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768184.842757:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775768184.842759:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768184.842761:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5562
00000100:00080000:2.0:1775768184.842764:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768184.842767:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768184.842775:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775768184.842783:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3480 x1862022114852736/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768184.842790:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768184.842792:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775768184.842794:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768184.842797:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5562
00000100:00080000:2.0:1775768184.842798:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768184.842800:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768184.842803:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775768184.842813:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114852608:192.168.202.110@tcp:250:kworker.0
00000100:00100000:2.0:1775768184.842818:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3100 x1862022114852864/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768184.842826:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768184.842828:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775768184.842829:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768184.842832:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5562
00000100:00080000:2.0:1775768184.842833:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768184.842836:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768184.842839:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775768184.842846:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2300 x1862022114852992/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768184.842852:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768184.842856:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1775768184.842857:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768184.842860:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5562
00000100:00080000:2.0:1775768184.842862:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768184.842864:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768184.842867:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775768184.842873:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0e00 x1862022114853120/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768184.842879:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768184.842881:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775768184.842883:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000800:00000100:0.0:1775768184.842885:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:2.0:1775768184.842889:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5562
00000100:00080000:2.0:1775768184.842892:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768184.842896:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768184.842900:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775768184.842908:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1880 x1862022114853248/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768184.842913:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768184.842916:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775768184.842917:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768184.842920:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5562
00000100:00080000:2.0:1775768184.842923:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768184.842925:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768184.842929:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775768184.842934:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768184/real 1775768184]  req@ffff9f14852d1c00 x1862022114852608/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768209 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775768184.842939:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1180 x1862022114853376/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768184.842966:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768184.842968:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775768184.842970:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775768184.842976:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1c00 x1862022114852608/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768209 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775768184.842976:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5562
00000100:00080000:2.0:1775768184.842983:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768184.842992:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00080000:0.0:1775768184.842996:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000008:00100000:2.0:1775768184.842998:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00080000:0.0:1775768184.843004:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768184.843009:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114852608:192.168.202.110@tcp:250:kworker.0
00000100:00100000:2.0:1775768184.843014:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0000 x1862022114853504/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775768184.843022:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114852736:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768184.843055:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768184.843069:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768184/real 1775768184]  req@ffff9f14852d3480 x1862022114852736/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768209 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768184.843083:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3480 x1862022114852736/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768209 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768184.843092:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768184.843096:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768184.843099:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114852736:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768184.843122:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114852864:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768184.843144:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768184.843156:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768184/real 1775768184]  req@ffff9f14852d3100 x1862022114852864/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768209 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768184.843182:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3100 x1862022114852864/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768209 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768184.843195:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768184.843199:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768184.843203:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114852864:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768184.843209:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114852992:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768184.843230:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768184.843242:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768184/real 1775768184]  req@ffff9f14852d2300 x1862022114852992/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768209 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768184.843254:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2300 x1862022114852992/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768209 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768184.843263:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768184.843266:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768184.843281:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114852992:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775768184.843292:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114853120:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768184.843312:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768184.843324:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768184/real 1775768184]  req@ffff9f14852d0e00 x1862022114853120/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768209 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768184.843353:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0e00 x1862022114853120/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768209 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768184.843362:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768184.843365:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768184.843369:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114853120:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775768184.843375:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114853248:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768184.843397:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768184.843408:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768184/real 1775768184]  req@ffff9f14852d1880 x1862022114853248/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768209 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768184.843421:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1880 x1862022114853248/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768209 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768184.843429:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768184.843433:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768184.843436:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114853248:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768184.843442:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114853376:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768184.843462:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768184.843474:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768184/real 1775768184]  req@ffff9f14852d1180 x1862022114853376/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768209 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768184.843486:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1180 x1862022114853376/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768209 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768184.843494:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768184.843498:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768184.843502:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114853376:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768184.843508:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114853504:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768184.843529:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768184.843541:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768184/real 1775768184]  req@ffff9f14852d0000 x1862022114853504/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768209 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768184.843580:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0000 x1862022114853504/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768209 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768184.843590:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768184.843594:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768184.843598:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114853504:192.168.202.110@tcp:8:kworker.0
00010000:00010000:3.0:1775768185.098725:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775768185.866661:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775768186.122759:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775768186.890664:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775768187.146734:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775768187.914653:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768188.170717:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775768188.938673:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768189.194728:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1775768189.962603:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768189.962611:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1775768189.962615:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00100000:1.0:1775768189.962618:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:2.0:1775768189.962624:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5567
00000100:00080000:2.0:1775768189.962633:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768189.962639:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775768189.962671:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2d80 x1862022114853632/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768189.962691:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768189.962693:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775768189.962695:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768189.962698:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5567
00000100:00080000:2.0:1775768189.962700:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768189.962702:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768189.962709:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775768189.962717:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0700 x1862022114853760/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768189.962724:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768189.962726:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775768189.962728:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775768189.962730:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114853632:192.168.202.110@tcp:250:kworker.0
00000100:00080000:2.0:1775768189.962732:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5567
00000100:00080000:2.0:1775768189.962736:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768189.962739:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768189.962742:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775768189.962748:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0380 x1862022114853888/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768189.962760:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768189.962762:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775768189.962764:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768189.962766:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5567
00000100:00080000:2.0:1775768189.962769:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768189.962771:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768189.962773:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775768189.962780:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3b80 x1862022114854016/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768189.962785:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768189.962787:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1775768189.962788:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000800:00000100:0.0:1775768189.962792:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:2.0:1775768189.962794:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5567
00000100:00080000:2.0:1775768189.962798:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768189.962800:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768189.962802:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775768189.962808:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3800 x1862022114854144/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768189.962813:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768189.962815:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775768189.962816:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768189.962819:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5567
00000100:00080000:2.0:1775768189.962821:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768189.962824:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768189.962826:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775768189.962834:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768189/real 1775768189]  req@ffff9f14852d2d80 x1862022114853632/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768214 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775768189.962834:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2680 x1862022114854272/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768189.962852:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768189.962856:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775768189.962858:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768189.962861:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5567
00000100:00080000:2.0:1775768189.962864:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1775768189.962866:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2d80 x1862022114853632/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768214 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775768189.962869:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768189.962874:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00080000:0.0:1775768189.962882:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00100000:2.0:1775768189.962883:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2a00 x1862022114854400/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775768189.962888:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768189.962894:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114853632:192.168.202.110@tcp:250:kworker.0
00000100:00080000:2.0:1775768189.962894:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768189.962901:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775768189.962903:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768189.962906:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5567
00000100:00080000:2.0:1775768189.962908:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768189.962910:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768189.962913:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775768189.962919:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114853760:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775768189.962921:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1500 x1862022114854528/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1775768189.962942:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768189.962951:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768189/real 1775768189]  req@ffff9f14852d0700 x1862022114853760/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768214 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768189.962958:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0700 x1862022114853760/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768214 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768189.962963:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768189.962965:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768189.962967:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114853760:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768189.962970:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114853888:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768189.962985:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768189.962990:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768189/real 1775768189]  req@ffff9f14852d0380 x1862022114853888/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768214 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768189.962996:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0380 x1862022114853888/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768214 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768189.963001:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768189.963002:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768189.963004:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114853888:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768189.963007:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114854016:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768189.963019:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768189.963027:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768189/real 1775768189]  req@ffff9f14852d3b80 x1862022114854016/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768214 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768189.963037:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3b80 x1862022114854016/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768214 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768189.963045:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768189.963048:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768189.963051:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114854016:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775768189.963056:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114854144:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768189.963074:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768189.963083:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768189/real 1775768189]  req@ffff9f14852d3800 x1862022114854144/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768214 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768189.963093:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3800 x1862022114854144/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768214 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768189.963102:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768189.963105:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768189.963108:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114854144:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775768189.963114:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114854272:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768189.963127:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768189.963136:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768189/real 1775768189]  req@ffff9f14852d2680 x1862022114854272/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768214 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768189.963147:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2680 x1862022114854272/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768214 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768189.963155:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768189.963157:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768189.963161:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114854272:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768189.963166:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114854400:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768189.963179:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768189.963186:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768189/real 1775768189]  req@ffff9f14852d2a00 x1862022114854400/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768214 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768189.963192:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2a00 x1862022114854400/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768214 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768189.963197:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768189.963198:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768189.963200:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114854400:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768189.963212:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114854528:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768189.963235:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768189.963245:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768189/real 1775768189]  req@ffff9f14852d1500 x1862022114854528/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768214 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768189.963254:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1500 x1862022114854528/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768214 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768189.963260:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768189.963263:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768189.963266:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114854528:192.168.202.110@tcp:8:kworker.0
00010000:00010000:3.0:1775768190.218687:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775768190.986659:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775768191.242702:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775768192.010691:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768192.266779:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775768193.034668:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775768193.290721:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775768194.058659:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775768194.314667:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775768195.082643:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:0.0:1775768195.082657:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775768195.082663:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1775768195.082668:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775768195.082677:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5572
00000100:00080000:0.0:1775768195.082688:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775768195.082694:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775768195.082734:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3100 x1862022114854656/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775768195.082755:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775768195.082758:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775768195.082760:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775768195.082764:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5572
00000100:00080000:0.0:1775768195.082767:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775768195.082771:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775768195.082781:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775768195.082793:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114854656:192.168.202.110@tcp:250:kworker.0
00000100:00100000:0.0:1775768195.082802:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3800 x1862022114854784/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775768195.082813:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775768195.082815:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:0.0:1775768195.082817:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775768195.082819:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5572
00000100:00080000:0.0:1775768195.082822:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775768195.082825:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775768195.082829:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775768195.082837:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d2300 x1862022114854912/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775768195.082847:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775768195.082849:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775768195.082850:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775768195.082852:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5572
00000100:00080000:0.0:1775768195.082854:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775768195.082856:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775768195.082858:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775768195.082864:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0a80 x1862022114855040/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775768195.082869:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775768195.082872:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1775768195.082873:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775768195.082875:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5572
00000100:00080000:0.0:1775768195.082876:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775768195.082879:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775768195.082882:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775768195.082888:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0e00 x1862022114855168/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775768195.082896:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:2.0:1775768195.082896:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114854784:192.168.202.110@tcp:38:kworker.0
00000100:00080000:0.0:1775768195.082901:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775768195.082904:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775768195.082907:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5572
00000100:00080000:0.0:1775768195.082910:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775768195.082913:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775768195.082917:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775768195.082924:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d1500 x1862022114855296/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775768195.082930:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775768195.082932:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:0.0:1775768195.082933:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:2.0:1775768195.082936:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114854912:192.168.202.110@tcp:38:kworker.0
00000100:00080000:0.0:1775768195.082938:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5572
00000100:00080000:0.0:1775768195.082944:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775768195.082947:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775768195.082950:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775768195.082959:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d2a00 x1862022114855424/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775768195.082965:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114855040:192.168.202.110@tcp:8:kworker.0
00000100:00080000:0.0:1775768195.082967:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775768195.082973:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775768195.082975:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775768195.082980:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5572
00000100:00080000:0.0:1775768195.082984:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775768195.082988:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775768195.082994:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775768195.082998:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114855168:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775768195.083009:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0000 x1862022114855552/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775768195.083024:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114855296:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775768195.083045:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114855424:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775768195.083064:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114855552:192.168.202.110@tcp:8:kworker.0
00010000:00010000:3.0:1775768195.338711:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775768196.106730:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768196.362680:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775768197.130686:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768197.386716:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775768198.154771:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000400:00000100:3.0:1775768198.154871:0:2393:0:(lib-socket.c:454:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.202.110:988
00000400:00000100:3.0:1775768198.154911:0:2393:0:(acceptor.c:111:lnet_connect_console_error()) Connection to 192.168.202.110@tcp at host 192.168.202.110 was unreachable: the network or that node may be down, or Lustre may be misconfigured.
00000800:00000100:3.0:1775768198.154928:0:2393:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:3.0:1775768198.154968:0:2393:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:3.0:1775768198.154977:0:2393:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:3.0:1775768198.154985:0:2393:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:3.0:1775768198.154993:0:2393:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:3.0:1775768198.155001:0:2393:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:3.0:1775768198.155009:0:2393:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:3.0:1775768198.155015:0:2393:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000100:00100000:2.0:1775768198.155084:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768195/real 1775768198]  req@ffff9f14a04d0000 x1862022114855552/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768220 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775768198.155110:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0000 x1862022114855552/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768220 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775768198.155123:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775768198.155128:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775768198.155133:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114855552:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775768198.155140:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768195/real 1775768198]  req@ffff9f14a04d0e00 x1862022114855168/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768220 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775768198.155151:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0e00 x1862022114855168/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768220 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775768198.155157:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775768198.155161:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775768198.155164:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114855168:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775768198.155171:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768195/real 1775768198]  req@ffff9f14a04d1500 x1862022114855296/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768220 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775768198.155181:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1500 x1862022114855296/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768220 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775768198.155189:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775768198.155192:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775768198.155195:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114855296:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775768198.155201:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768195/real 1775768198]  req@ffff9f14a04d2a00 x1862022114855424/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768220 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775768198.155224:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d2a00 x1862022114855424/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768220 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775768198.155231:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775768198.155234:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775768198.155237:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114855424:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775768198.155243:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768195/real 1775768198]  req@ffff9f14a04d3800 x1862022114854784/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768220 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775768198.155254:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3800 x1862022114854784/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768220 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775768198.155261:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775768198.155264:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775768198.155267:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114854784:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775768198.155273:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768195/real 1775768198]  req@ffff9f14a04d2300 x1862022114854912/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768220 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775768198.155283:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d2300 x1862022114854912/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768220 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775768198.155291:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775768198.155294:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775768198.155297:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114854912:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775768198.155303:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768195/real 1775768198]  req@ffff9f14a04d0a80 x1862022114855040/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768220 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775768198.155313:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0a80 x1862022114855040/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768220 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775768198.155319:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775768198.155322:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775768198.155325:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114855040:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775768198.155331:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768195/real 1775768198]  req@ffff9f14a04d3100 x1862022114854656/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768220 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775768198.155342:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3100 x1862022114854656/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768220 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775768198.155349:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775768198.155352:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775768198.155354:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114854656:192.168.202.110@tcp:250:kworker.0
00010000:00010000:1.0:1775768198.410730:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775768199.178704:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768199.434713:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1775768200.202712:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775768200.202724:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1775768200.202729:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775768200.202742:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5577
00000100:00080000:0.0:1775768200.202756:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775768200.202764:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775768200.202775:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:0.0:1775768200.202803:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d1180 x1862022114855680/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775768200.202845:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775768200.202850:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775768200.202852:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775768200.202857:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5577
00000100:00080000:0.0:1775768200.202860:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775768200.202865:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775768200.202875:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775768200.202887:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3b80 x1862022114855808/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775768200.202898:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775768200.202902:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:0.0:1775768200.202904:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775768200.202908:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5577
00000100:00080000:0.0:1775768200.202911:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775768200.202915:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775768200.202921:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775768200.202929:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d1f80 x1862022114855936/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775768200.202939:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775768200.202943:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775768200.202945:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775768200.202948:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5577
00000100:00080000:0.0:1775768200.202951:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775768200.202954:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775768200.202959:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775768200.202967:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0380 x1862022114856064/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775768200.202975:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775768200.202979:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1775768200.202980:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775768200.202983:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5577
00000100:00080000:0.0:1775768200.202986:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775768200.202989:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775768200.202994:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775768200.203003:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3480 x1862022114856192/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775768200.203013:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775768200.203017:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775768200.203020:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775768200.203024:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5577
00000100:00080000:0.0:1775768200.203028:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775768200.203032:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775768200.203038:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775768200.203048:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3100 x1862022114856320/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775768200.203057:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775768200.203061:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:0.0:1775768200.203063:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775768200.203066:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5577
00000100:00080000:0.0:1775768200.203069:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775768200.203072:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775768200.203076:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775768200.203084:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0a80 x1862022114856448/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775768200.203093:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775768200.203096:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775768200.203098:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775768200.203101:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5577
00000100:00080000:0.0:1775768200.203104:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775768200.203107:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775768200.203112:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775768200.203121:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d2300 x1862022114856576/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775768200.203440:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114855680:192.168.202.110@tcp:250:kworker.0
00000800:00000100:2.0:1775768200.203533:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775768200.203617:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768200/real 1775768200]  req@ffff9f14a04d1180 x1862022114855680/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768225 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775768200.203639:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1180 x1862022114855680/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768225 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775768200.203651:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775768200.203657:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775768200.203662:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114855680:192.168.202.110@tcp:250:kworker.0
00000100:00100000:2.0:1775768200.203670:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114855808:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775768200.203694:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775768200.203705:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768200/real 1775768200]  req@ffff9f14a04d3b80 x1862022114855808/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768225 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775768200.203718:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3b80 x1862022114855808/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768225 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775768200.203728:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775768200.203732:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775768200.203735:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114855808:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775768200.203741:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114855936:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775768200.203758:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775768200.203766:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768200/real 1775768200]  req@ffff9f14a04d1f80 x1862022114855936/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768225 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775768200.203778:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1f80 x1862022114855936/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768225 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775768200.203795:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775768200.203798:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775768200.203802:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114855936:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775768200.203807:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114856064:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775768200.203823:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775768200.203831:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768200/real 1775768200]  req@ffff9f14a04d0380 x1862022114856064/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768225 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775768200.203843:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0380 x1862022114856064/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768225 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775768200.203851:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775768200.203854:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775768200.203858:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114856064:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775768200.203863:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114856192:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775768200.203879:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775768200.203886:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768200/real 1775768200]  req@ffff9f14a04d3480 x1862022114856192/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768225 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775768200.203898:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3480 x1862022114856192/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768225 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775768200.203906:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775768200.203909:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775768200.203913:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114856192:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775768200.203919:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114856320:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775768200.203935:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775768200.203944:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768200/real 1775768200]  req@ffff9f14a04d3100 x1862022114856320/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768225 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775768200.203956:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3100 x1862022114856320/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768225 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775768200.203965:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775768200.203968:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775768200.203972:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114856320:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775768200.203977:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114856448:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775768200.203992:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775768200.204001:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768200/real 1775768200]  req@ffff9f14a04d0a80 x1862022114856448/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768225 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775768200.204012:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0a80 x1862022114856448/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768225 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775768200.204020:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775768200.204024:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775768200.204027:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114856448:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775768200.204034:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114856576:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775768200.204055:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775768200.204066:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768200/real 1775768200]  req@ffff9f14a04d2300 x1862022114856576/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768225 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775768200.204079:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d2300 x1862022114856576/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768225 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775768200.204088:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775768200.204091:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775768200.204095:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114856576:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775768200.458764:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775768201.226829:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775768201.482723:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775768202.250707:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768202.506767:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775768203.274646:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768203.530729:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775768204.298673:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768204.554803:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1775768205.322690:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:1.0:1775768205.322695:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:2.0:1775768205.322708:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1775768205.322713:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768205.322725:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5582
00000100:00080000:2.0:1775768205.322739:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768205.322747:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775768205.322794:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0a80 x1862022114856704/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768205.322827:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768205.322830:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775768205.322832:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768205.322836:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5582
00000100:00080000:2.0:1775768205.322840:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768205.322843:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768205.322854:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775768205.322863:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114856704:192.168.202.110@tcp:250:kworker.0
00000100:00100000:2.0:1775768205.322872:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1f80 x1862022114856832/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768205.322885:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768205.322889:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775768205.322891:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768205.322897:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5582
00000100:00080000:2.0:1775768205.322901:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768205.322906:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768205.322911:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775768205.322926:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1500 x1862022114856960/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768205.322934:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000800:00000100:0.0:1775768205.322939:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:2.0:1775768205.322939:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775768205.322943:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768205.322948:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5582
00000100:00080000:2.0:1775768205.322952:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768205.322955:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768205.322960:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775768205.322969:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2a00 x1862022114857088/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768205.322979:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768205.322983:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1775768205.322985:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775768205.322988:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768205/real 1775768205]  req@ffff9f14852d0a80 x1862022114856704/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768230 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775768205.322995:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5582
00000100:00080000:2.0:1775768205.323004:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768205.323010:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775768205.323012:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0a80 x1862022114856704/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768230 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:2.0:1775768205.323019:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00080000:0.0:1775768205.323033:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00100000:2.0:1775768205.323035:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2680 x1862022114857216/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775768205.323042:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768205.323052:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114856704:192.168.202.110@tcp:250:kworker.0
00000100:00080000:2.0:1775768205.323056:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768205.323062:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775768205.323064:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768205.323070:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5582
00000100:00080000:2.0:1775768205.323074:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768205.323078:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768205.323083:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775768205.323086:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114856832:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775768205.323095:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3800 x1862022114857344/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768205.323105:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768205.323108:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775768205.323109:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768205.323113:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5582
00000800:00000100:0.0:1775768205.323117:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:2.0:1775768205.323119:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768205.323123:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768205.323127:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775768205.323133:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768205/real 1775768205]  req@ffff9f14852d1f80 x1862022114856832/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768230 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775768205.323154:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3b80 x1862022114857472/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768205.323162:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768205.323165:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775768205.323167:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768205.323173:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5582
00000100:00080000:2.0:1775768205.323179:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768205.323185:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768205.323189:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775768205.323196:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1f80 x1862022114856832/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768230 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775768205.323200:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0380 x1862022114857600/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775768205.323229:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768205.323235:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768205.323240:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114856832:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768205.323249:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114856960:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768205.323271:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768205.323280:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768205/real 1775768205]  req@ffff9f14852d1500 x1862022114856960/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768230 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768205.323289:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1500 x1862022114856960/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768230 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768205.323294:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768205.323297:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768205.323299:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114856960:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768205.323303:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114857088:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768205.323314:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768205.323320:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768205/real 1775768205]  req@ffff9f14852d2a00 x1862022114857088/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768230 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768205.323327:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2a00 x1862022114857088/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768230 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768205.323332:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768205.323334:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768205.323336:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114857088:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775768205.323339:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114857216:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768205.323349:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768205.323355:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768205/real 1775768205]  req@ffff9f14852d2680 x1862022114857216/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768230 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768205.323378:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2680 x1862022114857216/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768230 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768205.323387:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768205.323389:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768205.323391:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114857216:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775768205.323406:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114857344:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768205.323419:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768205.323426:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768205/real 1775768205]  req@ffff9f14852d3800 x1862022114857344/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768230 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768205.323434:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3800 x1862022114857344/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768230 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768205.323438:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768205.323440:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768205.323442:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114857344:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768205.323445:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114857472:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768205.323456:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768205.323461:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768205/real 1775768205]  req@ffff9f14852d3b80 x1862022114857472/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768230 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768205.323468:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3b80 x1862022114857472/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768230 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768205.323478:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768205.323480:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768205.323482:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114857472:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768205.323486:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114857600:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768205.323495:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768205.323500:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768205/real 1775768205]  req@ffff9f14852d0380 x1862022114857600/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768230 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768205.323509:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0380 x1862022114857600/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768230 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768205.323516:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768205.323519:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768205.323523:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114857600:192.168.202.110@tcp:8:kworker.0
00010000:00010000:1.0:1775768205.578764:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775768206.346685:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768206.602776:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775768207.370649:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775768207.626774:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775768208.394695:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775768208.650721:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775768209.418621:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768209.674659:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1775768210.442719:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775768210.442726:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1775768210.442730:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775768210.442739:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5587
00000100:00080000:1.0:1775768210.442750:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775768210.442756:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775768210.442790:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff800 x1862022114857728/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775768210.442812:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775768210.442815:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775768210.442817:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775768210.442819:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5587
00000100:00080000:1.0:1775768210.442822:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775768210.442824:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775768210.442832:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775768210.442841:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fed80 x1862022114857856/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775768210.442849:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775768210.442851:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775768210.442853:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775768210.442857:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5587
00000100:00100000:0.0:1775768210.442858:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114857728:192.168.202.110@tcp:250:kworker.0
00000100:00080000:1.0:1775768210.442859:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775768210.442864:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775768210.442867:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775768210.442873:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ffb80 x1862022114857984/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775768210.442880:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775768210.442882:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775768210.442884:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775768210.442886:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5587
00000100:00080000:1.0:1775768210.442889:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775768210.442891:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775768210.442894:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775768210.442905:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fca80 x1862022114858112/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775768210.442914:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775768210.442917:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1775768210.442919:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775768210.442922:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5587
00000100:00080000:1.0:1775768210.442927:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775768210.442930:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775768210.442936:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:0.0:1775768210.442937:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775768210.442953:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fc380 x1862022114858240/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775768210.442965:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775768210.442968:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775768210.442970:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775768210.442973:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5587
00000100:00080000:1.0:1775768210.442978:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775768210.442983:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775768210.442987:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768210/real 1775768210]  req@ffff9f14ab4ff800 x1862022114857728/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768235 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:1.0:1775768210.442991:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775768210.443010:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff100 x1862022114858368/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775768210.443016:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff800 x1862022114857728/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768235 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775768210.443034:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775768210.443039:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775768210.443041:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775768210.443045:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5587
00000100:00080000:0.0:1775768210.443048:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775768210.443049:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775768210.443056:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00080000:0.0:1775768210.443058:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768210.443063:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114857728:192.168.202.110@tcp:250:kworker.0
00000008:00100000:1.0:1775768210.443063:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775768210.443077:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fce00 x1862022114858496/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775768210.443101:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775768210.443105:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775768210.443106:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775768210.443109:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5587
00000100:00080000:1.0:1775768210.443114:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1775768210.443116:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114857856:192.168.202.110@tcp:38:kworker.0
00000100:00080000:1.0:1775768210.443120:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775768210.443126:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775768210.443141:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdc00 x1862022114858624/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1775768210.443156:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768210.443168:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768210/real 1775768210]  req@ffff9f14ab4fed80 x1862022114857856/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768235 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768210.443179:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fed80 x1862022114857856/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768235 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768210.443185:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768210.443188:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768210.443190:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114857856:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768210.443195:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ffb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114857984:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768210.443209:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775768210.443213:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:0.0:1775768210.443218:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768210/real 1775768210]  req@ffff9f14ab4ffb80 x1862022114857984/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768235 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768210.443227:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ffb80 x1862022114857984/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768235 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768210.443234:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768210.443236:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768210.443238:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ffb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114857984:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768210.443243:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114858112:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768210.443254:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768210.443259:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768210/real 1775768210]  req@ffff9f14ab4fca80 x1862022114858112/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768235 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768210.443267:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fca80 x1862022114858112/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768235 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768210.443272:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768210.443274:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768210.443276:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114858112:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775768210.443279:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114858240:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768210.443289:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768210.443294:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768210/real 1775768210]  req@ffff9f14ab4fc380 x1862022114858240/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768235 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768210.443301:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fc380 x1862022114858240/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768235 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768210.443306:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768210.443308:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768210.443310:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114858240:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775768210.443313:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114858368:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768210.443328:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768210.443337:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768210/real 1775768210]  req@ffff9f14ab4ff100 x1862022114858368/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768235 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768210.443350:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff100 x1862022114858368/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768235 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768210.443366:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768210.443370:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768210.443373:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114858368:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768210.443379:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114858496:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768210.443404:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768210.443412:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768210/real 1775768210]  req@ffff9f14ab4fce00 x1862022114858496/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768235 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768210.443420:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fce00 x1862022114858496/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768235 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768210.443425:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768210.443427:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768210.443429:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114858496:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768210.443443:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114858624:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768210.443454:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768210.443464:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768210/real 1775768210]  req@ffff9f14ab4fdc00 x1862022114858624/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768235 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768210.443476:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdc00 x1862022114858624/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768235 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768210.443485:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768210.443488:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768210.443492:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114858624:192.168.202.110@tcp:8:kworker.0
00010000:00010000:3.0:1775768210.698735:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768211.466671:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768211.722697:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768212.490646:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775768212.746714:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768213.514661:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768213.770679:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775768214.538667:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775768214.794711:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1775768215.562668:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:1.0:1775768215.562669:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:0.0:1775768215.562678:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1775768215.562682:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775768215.562692:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5593
00000100:00080000:0.0:1775768215.562703:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775768215.562710:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775768215.562748:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3800 x1862022114858752/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775768215.562775:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775768215.562779:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775768215.562780:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775768215.562784:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5593
00000100:00080000:0.0:1775768215.562787:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775768215.562791:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775768215.562801:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775768215.562813:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d2a00 x1862022114858880/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775768215.562819:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114858752:192.168.202.110@tcp:250:kworker.0
00000100:00080000:0.0:1775768215.562829:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775768215.562832:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:0.0:1775768215.562834:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775768215.562838:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5593
00000100:00080000:0.0:1775768215.562842:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775768215.562846:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775768215.562850:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775768215.562861:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d1500 x1862022114859008/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775768215.562869:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775768215.562873:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775768215.562875:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775768215.562878:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5593
00000100:00080000:0.0:1775768215.562882:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775768215.562888:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000800:00000100:2.0:1775768215.562890:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000008:00100000:0.0:1775768215.562895:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775768215.562907:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0e00 x1862022114859136/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775768215.562916:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775768215.562919:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1775768215.562921:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775768215.562924:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5593
00000100:00080000:0.0:1775768215.562928:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00100000:2.0:1775768215.562933:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768215/real 1775768215]  req@ffff9f14a04d3800 x1862022114858752/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768240 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768215.562934:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775768215.562941:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775768215.562955:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0000 x1862022114859264/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775768215.562959:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3800 x1862022114858752/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768240 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768215.562976:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768215.562980:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768215.562981:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775768215.562984:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768215.562988:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775768215.562993:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114858752:192.168.202.110@tcp:250:kworker.0
00000100:00080000:0.0:1775768215.562998:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5593
00000100:00080000:0.0:1775768215.563002:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775768215.563006:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775768215.563011:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775768215.563019:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114858880:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768215.563024:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3800 x1862022114859392/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775768215.563047:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775768215.563051:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:0.0:1775768215.563053:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775768215.563058:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5593
00000800:00000100:2.0:1775768215.563060:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:0.0:1775768215.563065:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775768215.563070:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775768215.563076:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775768215.563079:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768215/real 1775768215]  req@ffff9f14a04d2a00 x1862022114858880/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768240 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768215.563089:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d2300 x1862022114859520/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775768215.563096:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d2a00 x1862022114858880/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768240 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768215.563103:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775768215.563111:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775768215.563114:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768215.563114:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768215.563122:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5593
00000100:00080000:2.0:1775768215.563124:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00080000:0.0:1775768215.563129:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:2.0:1775768215.563130:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114858880:192.168.202.110@tcp:38:kworker.0
00000100:00080000:0.0:1775768215.563137:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775768215.563141:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114859008:192.168.202.110@tcp:38:kworker.0
00000008:00100000:0.0:1775768215.563143:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775768215.563156:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0a80 x1862022114859648/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:2.0:1775768215.563167:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775768215.563181:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768215/real 1775768215]  req@ffff9f14a04d1500 x1862022114859008/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768240 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775768215.563193:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1500 x1862022114859008/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768240 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775768215.563201:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775768215.563204:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775768215.563207:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114859008:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775768215.563212:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114859136:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775768215.563229:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775768215.563239:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768215/real 1775768215]  req@ffff9f14a04d0e00 x1862022114859136/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768240 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775768215.563251:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0e00 x1862022114859136/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768240 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775768215.563258:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775768215.563261:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775768215.563263:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114859136:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775768215.563269:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114859264:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775768215.563284:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775768215.563292:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768215/real 1775768215]  req@ffff9f14a04d0000 x1862022114859264/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768240 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775768215.563302:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0000 x1862022114859264/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768240 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775768215.563309:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775768215.563313:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775768215.563316:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114859264:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775768215.563333:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114859392:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775768215.563349:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775768215.563358:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768215/real 1775768215]  req@ffff9f14a04d3800 x1862022114859392/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768240 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775768215.563368:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3800 x1862022114859392/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768240 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775768215.563375:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775768215.563378:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775768215.563381:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114859392:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775768215.563386:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114859520:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775768215.563403:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775768215.563412:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768215/real 1775768215]  req@ffff9f14a04d2300 x1862022114859520/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768240 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775768215.563422:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d2300 x1862022114859520/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768240 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775768215.563429:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775768215.563432:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775768215.563435:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114859520:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775768215.563440:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114859648:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775768215.563455:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775768215.563464:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768215/real 1775768215]  req@ffff9f14a04d0a80 x1862022114859648/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768240 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775768215.563475:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0a80 x1862022114859648/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768240 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775768215.563482:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775768215.563485:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775768215.563488:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114859648:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775768215.818715:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775768216.586620:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768216.842690:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775768217.610655:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768217.866746:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775768218.634673:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768218.890699:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775768219.658689:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775768219.914766:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1775768220.682685:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:1.0:1775768220.682689:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:0.0:1775768220.682698:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1775768220.682702:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775768220.682714:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5598
00000100:00080000:0.0:1775768220.682727:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775768220.682735:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775768220.682776:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3100 x1862022114859776/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775768220.682806:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775768220.682810:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775768220.682811:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775768220.682816:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5598
00000100:00080000:0.0:1775768220.682819:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775768220.682822:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775768220.682832:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775768220.682847:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3480 x1862022114859904/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775768220.682859:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775768220.682863:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:0.0:1775768220.682864:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:2.0:1775768220.682872:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114859776:192.168.202.110@tcp:250:kworker.0
00000100:00080000:0.0:1775768220.682873:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5598
00000100:00080000:0.0:1775768220.682883:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775768220.682888:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775768220.682894:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775768220.682908:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0380 x1862022114860032/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775768220.682919:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775768220.682922:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775768220.682924:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775768220.682929:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5598
00000100:00080000:0.0:1775768220.682941:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775768220.682945:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775768220.682950:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:2.0:1775768220.682954:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768220.682963:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d1f80 x1862022114860160/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775768220.682972:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775768220.682975:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1775768220.682978:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775768220.682981:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5598
00000100:00080000:0.0:1775768220.682995:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775768220.682999:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775768220.683005:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775768220.683015:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3b80 x1862022114860288/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775768220.683023:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768220/real 1775768220]  req@ffff9f14a04d3100 x1862022114859776/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768245 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768220.683046:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775768220.683050:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775768220.683052:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775768220.683057:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5598
00000100:00080000:0.0:1775768220.683062:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:2.0:1775768220.683062:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3100 x1862022114859776/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768245 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768220.683070:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775768220.683075:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00080000:2.0:1775768220.683086:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00100000:0.0:1775768220.683088:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d1180 x1862022114860416/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768220.683095:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:0.0:1775768220.683103:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:2.0:1775768220.683104:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114859776:192.168.202.110@tcp:250:kworker.0
00000100:00080000:0.0:1775768220.683110:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:0.0:1775768220.683113:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775768220.683118:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5598
00000100:00080000:0.0:1775768220.683121:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775768220.683125:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775768220.683130:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775768220.683137:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114859904:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768220.683143:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3100 x1862022114860544/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775768220.683153:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775768220.683157:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775768220.683159:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775768220.683163:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5598
00000100:00080000:0.0:1775768220.683168:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000800:00000100:2.0:1775768220.683171:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:0.0:1775768220.683175:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775768220.683180:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775768220.683186:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768220/real 1775768220]  req@ffff9f14a04d3480 x1862022114859904/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768245 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768220.683194:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0a80 x1862022114860672/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775768220.683203:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3480 x1862022114859904/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768245 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775768220.683213:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775768220.683217:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775768220.683221:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114859904:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775768220.683228:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114860032:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775768220.683251:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775768220.683260:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768220/real 1775768220]  req@ffff9f14a04d0380 x1862022114860032/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768245 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775768220.683271:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0380 x1862022114860032/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768245 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775768220.683278:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775768220.683281:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775768220.683283:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114860032:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775768220.683289:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114860160:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775768220.683307:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775768220.683316:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768220/real 1775768220]  req@ffff9f14a04d1f80 x1862022114860160/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768245 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775768220.683327:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1f80 x1862022114860160/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768245 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775768220.683336:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775768220.683339:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775768220.683342:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114860160:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775768220.683347:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114860288:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775768220.683361:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775768220.683369:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768220/real 1775768220]  req@ffff9f14a04d3b80 x1862022114860288/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768245 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775768220.683386:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3b80 x1862022114860288/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768245 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775768220.683393:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775768220.683396:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775768220.683399:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114860288:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775768220.683404:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114860416:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775768220.683418:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775768220.683426:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768220/real 1775768220]  req@ffff9f14a04d1180 x1862022114860416/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768245 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775768220.683437:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1180 x1862022114860416/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768245 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775768220.683444:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775768220.683447:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775768220.683450:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114860416:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775768220.683468:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114860544:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775768220.683482:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775768220.683490:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768220/real 1775768220]  req@ffff9f14a04d3100 x1862022114860544/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768245 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775768220.683500:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3100 x1862022114860544/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768245 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775768220.683513:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775768220.683516:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775768220.683519:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114860544:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775768220.683524:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114860672:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775768220.683539:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775768220.683548:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768220/real 1775768220]  req@ffff9f14a04d0a80 x1862022114860672/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768245 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775768220.683587:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0a80 x1862022114860672/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768245 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775768220.683595:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775768220.683598:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775768220.683601:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114860672:192.168.202.110@tcp:8:kworker.0
00010000:00010000:3.0:1775768220.938664:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775768221.706634:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775768221.962755:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775768222.730661:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775768222.986691:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775768223.754660:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768224.010699:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775768224.778632:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768225.034706:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775768225.802658:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:2.0:1775768225.802731:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768225.802737:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1775768225.802741:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768225.802750:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5603
00000100:00080000:2.0:1775768225.802760:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768225.802767:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775768225.802810:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0700 x1862022114860800/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768225.802837:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768225.802840:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775768225.802841:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768225.802845:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5603
00000100:00080000:2.0:1775768225.802849:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768225.802853:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768225.802862:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775768225.802870:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114860800:192.168.202.110@tcp:250:kworker.0
00000100:00100000:2.0:1775768225.802877:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2d80 x1862022114860928/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768225.802887:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768225.802890:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775768225.802892:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768225.802897:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5603
00000100:00080000:2.0:1775768225.802901:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768225.802903:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768225.802906:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775768225.802915:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0000 x1862022114861056/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1775768225.802920:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:2.0:1775768225.802929:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768225.802932:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775768225.802934:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768225.802963:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5603
00000100:00100000:0.0:1775768225.802964:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768225/real 1775768225]  req@ffff9f14852d0700 x1862022114860800/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768250 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775768225.802970:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768225.802978:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775768225.802983:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0700 x1862022114860800/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768250 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:2.0:1775768225.802983:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00080000:0.0:1775768225.802994:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00100000:2.0:1775768225.802998:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1180 x1862022114861184/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775768225.803001:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768225.803007:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114860800:192.168.202.110@tcp:250:kworker.0
00000100:00080000:2.0:1775768225.803010:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768225.803013:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1775768225.803014:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768225.803018:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5603
00000100:00080000:2.0:1775768225.803021:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768225.803023:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768225.803026:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775768225.803030:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114860928:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775768225.803035:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1880 x1862022114861312/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768225.803043:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768225.803045:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775768225.803047:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768225.803049:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5603
00000100:00080000:2.0:1775768225.803052:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768225.803055:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768225.803057:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775768225.803062:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0e00 x1862022114861440/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1775768225.803069:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:2.0:1775768225.803080:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768225.803083:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00100000:0.0:1775768225.803086:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768225/real 1775768225]  req@ffff9f14852d2d80 x1862022114860928/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768250 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775768225.803086:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768225.803095:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5603
00000100:00080000:2.0:1775768225.803100:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1775768225.803103:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2d80 x1862022114860928/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768250 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775768225.803107:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768225.803115:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00080000:0.0:1775768225.803116:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768225.803121:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768225.803125:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114860928:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775768225.803127:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2300 x1862022114861568/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775768225.803133:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114861056:192.168.202.110@tcp:38:kworker.0
00000100:00080000:2.0:1775768225.803139:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768225.803141:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775768225.803143:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768225.803146:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5603
00000100:00080000:2.0:1775768225.803149:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768225.803152:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000800:00000100:0.0:1775768225.803157:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000008:00100000:2.0:1775768225.803159:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775768225.803168:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3100 x1862022114861696/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775768225.803174:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768225/real 1775768225]  req@ffff9f14852d0000 x1862022114861056/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768250 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768225.803190:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0000 x1862022114861056/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768250 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768225.803198:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768225.803201:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768225.803204:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114861056:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768225.803211:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114861184:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768225.803229:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768225.803236:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768225/real 1775768225]  req@ffff9f14852d1180 x1862022114861184/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768250 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768225.803243:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1180 x1862022114861184/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768250 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768225.803248:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768225.803250:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768225.803251:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114861184:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775768225.803262:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114861312:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768225.803271:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768225.803277:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768225/real 1775768225]  req@ffff9f14852d1880 x1862022114861312/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768250 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768225.803283:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1880 x1862022114861312/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768250 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768225.803287:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768225.803289:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768225.803290:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114861312:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775768225.803293:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114861440:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768225.803302:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768225.803312:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768225/real 1775768225]  req@ffff9f14852d0e00 x1862022114861440/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768250 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768225.803318:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0e00 x1862022114861440/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768250 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768225.803323:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768225.803324:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768225.803326:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114861440:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768225.803329:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114861568:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768225.803337:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768225.803342:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768225/real 1775768225]  req@ffff9f14852d2300 x1862022114861568/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768250 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768225.803348:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2300 x1862022114861568/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768250 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768225.803352:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768225.803354:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768225.803356:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114861568:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768225.803358:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114861696:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768225.803366:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768225.803371:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768225/real 1775768225]  req@ffff9f14852d3100 x1862022114861696/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768250 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768225.803381:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3100 x1862022114861696/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768250 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768225.803387:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768225.803390:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768225.803393:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114861696:192.168.202.110@tcp:8:kworker.0
00010000:00010000:3.0:1775768226.058712:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775768226.826631:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775768227.082721:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775768227.850672:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768228.106707:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775768228.874630:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775768229.130697:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775768229.898648:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775768230.154715:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1775768230.922660:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:2.0:1775768230.922664:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:1.0:1775768230.922668:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1775768230.922673:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775768230.922681:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5608
00000100:00080000:1.0:1775768230.922690:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775768230.922695:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775768230.922732:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fea00 x1862022114861824/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775768230.922760:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775768230.922764:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775768230.922766:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775768230.922770:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5608
00000100:00080000:1.0:1775768230.922773:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775768230.922777:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775768230.922789:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775768230.922799:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdf80 x1862022114861952/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775768230.922807:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114861824:192.168.202.110@tcp:250:kworker.0
00000100:00080000:1.0:1775768230.922808:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775768230.922814:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775768230.922817:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775768230.922820:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5608
00000100:00080000:1.0:1775768230.922822:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775768230.922824:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775768230.922828:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775768230.922836:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fe300 x1862022114862080/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775768230.922842:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775768230.922845:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775768230.922846:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775768230.922848:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5608
00000100:00080000:1.0:1775768230.922860:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775768230.922862:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775768230.922865:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:0.0:1775768230.922870:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775768230.922874:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fd880 x1862022114862208/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775768230.922880:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775768230.922883:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1775768230.922884:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775768230.922902:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5608
00000100:00080000:1.0:1775768230.922905:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775768230.922908:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775768230.922911:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775768230.922919:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdc00 x1862022114862336/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775768230.922926:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1775768230.922927:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768230/real 1775768230]  req@ffff9f14ab4fea00 x1862022114861824/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768255 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775768230.922930:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775768230.922933:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775768230.922939:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5608
00000100:00080000:1.0:1775768230.922943:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775768230.922948:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775768230.922951:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775768230.922956:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fea00 x1862022114861824/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768255 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775768230.922959:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fce00 x1862022114862464/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775768230.922979:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775768230.922980:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775768230.922986:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:1.0:1775768230.922986:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775768230.922990:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775768230.922993:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114861824:192.168.202.110@tcp:250:kworker.0
00000100:00080000:1.0:1775768230.922997:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5608
00000100:00080000:1.0:1775768230.923003:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775768230.923007:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775768230.923013:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775768230.923021:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff100 x1862022114862592/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775768230.923024:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114861952:192.168.202.110@tcp:38:kworker.0
00000100:00080000:1.0:1775768230.923035:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775768230.923037:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775768230.923039:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775768230.923042:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5608
00000800:00000100:0.0:1775768230.923053:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:1.0:1775768230.923057:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775768230.923061:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775768230.923065:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775768230.923070:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768230/real 1775768230]  req@ffff9f14ab4fdf80 x1862022114861952/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768255 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775768230.923075:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fc380 x1862022114862720/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775768230.923098:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdf80 x1862022114861952/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768255 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768230.923107:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768230.923110:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768230.923113:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114861952:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768230.923119:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114862080:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768230.923138:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768230.923148:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768230/real 1775768230]  req@ffff9f14ab4fe300 x1862022114862080/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768255 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768230.923158:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fe300 x1862022114862080/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768255 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768230.923164:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768230.923167:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768230.923170:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114862080:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768230.923175:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114862208:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768230.923190:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768230.923197:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768230/real 1775768230]  req@ffff9f14ab4fd880 x1862022114862208/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768255 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768230.923207:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fd880 x1862022114862208/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768255 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768230.923213:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768230.923216:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768230.923219:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114862208:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775768230.923223:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114862336:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768230.923236:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768230.923243:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768230/real 1775768230]  req@ffff9f14ab4fdc00 x1862022114862336/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768255 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768230.923253:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdc00 x1862022114862336/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768255 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768230.923259:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768230.923261:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768230.923263:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114862336:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775768230.923268:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114862464:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768230.923283:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768230.923291:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768230/real 1775768230]  req@ffff9f14ab4fce00 x1862022114862464/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768255 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768230.923300:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fce00 x1862022114862464/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768255 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768230.923312:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768230.923314:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768230.923317:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114862464:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768230.923333:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114862592:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768230.923362:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768230.923372:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768230/real 1775768230]  req@ffff9f14ab4ff100 x1862022114862592/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768255 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768230.923382:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff100 x1862022114862592/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768255 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768230.923389:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768230.923392:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768230.923395:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114862592:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768230.923400:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114862720:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768230.923417:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768230.923426:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768230/real 1775768230]  req@ffff9f14ab4fc380 x1862022114862720/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768255 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768230.923435:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fc380 x1862022114862720/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768255 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768230.923441:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768230.923444:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768230.923447:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114862720:192.168.202.110@tcp:8:kworker.0
00010000:00010000:3.0:1775768231.178734:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768231.946651:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768232.202754:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768232.970693:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768233.226679:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768233.994641:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775768234.250701:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768235.018645:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768235.274687:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775768236.042662:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:2.0:1775768236.042682:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768236.042689:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1775768236.042694:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768236.042705:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5613
00000100:00080000:2.0:1775768236.042715:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768236.042722:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775768236.042756:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3480 x1862022114862848/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768236.042779:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768236.042782:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775768236.042784:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768236.042789:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5613
00000100:00080000:2.0:1775768236.042794:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768236.042798:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768236.042809:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775768236.042821:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114862848:192.168.202.110@tcp:250:kworker.0
00000100:00100000:2.0:1775768236.042829:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1c00 x1862022114862976/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768236.042853:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768236.042857:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775768236.042860:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768236.042865:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5613
00000100:00080000:2.0:1775768236.042869:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768236.042874:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768236.042878:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000800:00000100:0.0:1775768236.042891:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775768236.042893:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3100 x1862022114863104/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768236.042903:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768236.042906:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775768236.042908:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768236.042911:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5613
00000100:00080000:2.0:1775768236.042916:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768236.042919:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775768236.042924:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768236/real 1775768236]  req@ffff9f14852d3480 x1862022114862848/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768261 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:2.0:1775768236.042925:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775768236.042945:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2300 x1862022114863232/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775768236.042947:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3480 x1862022114862848/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768261 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768236.042971:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775768236.042971:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775768236.042978:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:2.0:1775768236.042978:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1775768236.042982:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775768236.042984:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114862848:192.168.202.110@tcp:250:kworker.0
00000100:00080000:2.0:1775768236.042988:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5613
00000100:00080000:2.0:1775768236.042992:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768236.042996:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768236.043000:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775768236.043011:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0e00 x1862022114863360/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775768236.043014:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114862976:192.168.202.110@tcp:38:kworker.0
00000100:00080000:2.0:1775768236.043033:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000800:00000100:0.0:1775768236.043036:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:2.0:1775768236.043039:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775768236.043041:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768236.043047:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5613
00000100:00100000:0.0:1775768236.043048:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768236/real 1775768236]  req@ffff9f14852d1c00 x1862022114862976/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768261 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775768236.043054:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768236.043062:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775768236.043063:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1c00 x1862022114862976/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768261 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:2.0:1775768236.043071:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00080000:0.0:1775768236.043072:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768236.043078:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768236.043080:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114862976:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768236.043084:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114863104:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775768236.043088:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1880 x1862022114863488/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768236.043097:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768236.043100:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000800:00000100:0.0:1775768236.043103:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:2.0:1775768236.043104:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768236.043109:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5613
00000100:00080000:2.0:1775768236.043113:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768236.043118:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768236.043123:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775768236.043128:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768236/real 1775768236]  req@ffff9f14852d3100 x1862022114863104/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768261 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768236.043143:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3100 x1862022114863104/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768261 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775768236.043150:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1180 x1862022114863616/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775768236.043154:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775768236.043165:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775768236.043167:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768236.043172:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114863104:192.168.202.110@tcp:38:kworker.0
00000100:00080000:2.0:1775768236.043172:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775768236.043176:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775768236.043180:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114863232:192.168.202.110@tcp:8:kworker.0
00000100:00080000:2.0:1775768236.043184:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5613
00000100:00080000:2.0:1775768236.043189:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768236.043195:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768236.043200:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:0.0:1775768236.043201:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768236.043212:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768236/real 1775768236]  req@ffff9f14852d2300 x1862022114863232/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768261 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775768236.043216:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0000 x1862022114863744/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775768236.043226:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2300 x1862022114863232/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768261 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768236.043236:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768236.043240:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768236.043246:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114863232:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775768236.043269:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114863360:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768236.043295:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768236.043319:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768236/real 1775768236]  req@ffff9f14852d0e00 x1862022114863360/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768261 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768236.043338:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0e00 x1862022114863360/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768261 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768236.043348:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768236.043354:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768236.043361:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114863360:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775768236.043367:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114863488:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768236.043387:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768236.043413:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768236/real 1775768236]  req@ffff9f14852d1880 x1862022114863488/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768261 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768236.043433:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1880 x1862022114863488/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768261 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768236.043439:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768236.043443:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768236.043446:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114863488:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768236.043451:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114863616:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768236.043472:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768236.043481:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768236/real 1775768236]  req@ffff9f14852d1180 x1862022114863616/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768261 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768236.043490:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1180 x1862022114863616/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768261 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768236.043502:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768236.043504:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768236.043507:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114863616:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768236.043519:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114863744:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768236.043541:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768236.043581:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768236/real 1775768236]  req@ffff9f14852d0000 x1862022114863744/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768261 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768236.043593:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0000 x1862022114863744/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768261 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768236.043601:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768236.043605:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768236.043609:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114863744:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775768236.298692:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775768237.066652:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775768237.322740:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775768238.090665:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768238.346848:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775768239.114653:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768239.370706:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775768240.138655:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768240.394699:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1775768241.162675:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:2.0:1775768241.162675:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:1.0:1775768241.162684:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1775768241.162689:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775768241.162699:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5618
00000100:00080000:1.0:1775768241.162709:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775768241.162717:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775768241.162756:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fca80 x1862022114863872/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775768241.162803:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775768241.162806:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775768241.162808:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775768241.162812:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5618
00000100:00080000:1.0:1775768241.162817:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775768241.162825:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775768241.162835:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775768241.162842:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114863872:192.168.202.110@tcp:250:kworker.0
00000100:00100000:1.0:1775768241.162852:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ffb80 x1862022114864000/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775768241.162862:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775768241.162866:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775768241.162867:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775768241.162871:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5618
00000100:00080000:1.0:1775768241.162875:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775768241.162879:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775768241.162884:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775768241.162895:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fed80 x1862022114864128/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1775768241.162903:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:1.0:1775768241.162910:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775768241.162914:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775768241.162916:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775768241.162921:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5618
00000100:00080000:1.0:1775768241.162925:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775768241.162928:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775768241.162933:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775768241.162944:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff800 x1862022114864256/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775768241.162956:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768241/real 1775768241]  req@ffff9f14ab4fca80 x1862022114863872/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768266 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775768241.162982:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775768241.162985:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00100000:0.0:1775768241.162988:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fca80 x1862022114863872/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768266 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775768241.162989:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775768241.162997:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5618
00000100:00080000:1.0:1775768241.163003:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775768241.163011:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00080000:0.0:1775768241.163013:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000008:00100000:1.0:1775768241.163016:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00080000:0.0:1775768241.163027:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775768241.163030:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fc380 x1862022114864384/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775768241.163034:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114863872:192.168.202.110@tcp:250:kworker.0
00000100:00080000:1.0:1775768241.163048:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775768241.163051:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775768241.163053:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775768241.163062:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ffb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114864000:192.168.202.110@tcp:38:kworker.0
00000100:00080000:1.0:1775768241.163067:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5618
00000100:00080000:1.0:1775768241.163072:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775768241.163077:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000800:00000100:0.0:1775768241.163082:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000008:00100000:1.0:1775768241.163083:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775768241.163092:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768241/real 1775768241]  req@ffff9f14ab4ffb80 x1862022114864000/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768266 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775768241.163098:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff100 x1862022114864512/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775768241.163105:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ffb80 x1862022114864000/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768266 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775768241.163117:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775768241.163119:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775768241.163123:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:0.0:1775768241.163126:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00080000:1.0:1775768241.163127:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775768241.163132:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ffb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114864000:192.168.202.110@tcp:38:kworker.0
00000100:00080000:1.0:1775768241.163136:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5618
00000100:00100000:0.0:1775768241.163141:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114864128:192.168.202.110@tcp:38:kworker.0
00000100:00080000:1.0:1775768241.163142:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775768241.163150:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775768241.163155:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775768241.163167:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fce00 x1862022114864640/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1775768241.163170:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:1.0:1775768241.163180:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1775768241.163183:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768241/real 1775768241]  req@ffff9f14ab4fed80 x1862022114864128/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768266 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775768241.163185:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775768241.163189:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775768241.163198:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5618
00000100:00100000:0.0:1775768241.163199:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fed80 x1862022114864128/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768266 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775768241.163204:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775768241.163206:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768241.163211:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00080000:1.0:1775768241.163211:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775768241.163217:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114864128:192.168.202.110@tcp:38:kworker.0
00000008:00100000:1.0:1775768241.163220:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775768241.163223:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114864256:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775768241.163237:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdc00 x1862022114864768/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1775768241.163246:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768241.163273:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768241/real 1775768241]  req@ffff9f14ab4ff800 x1862022114864256/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768266 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768241.163286:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff800 x1862022114864256/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768266 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768241.163292:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768241.163294:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768241.163296:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114864256:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775768241.163306:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114864384:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768241.163319:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768241.163325:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768241/real 1775768241]  req@ffff9f14ab4fc380 x1862022114864384/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768266 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768241.163332:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fc380 x1862022114864384/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768266 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768241.163336:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768241.163338:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768241.163340:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114864384:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775768241.163350:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114864512:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768241.163361:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768241.163369:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768241/real 1775768241]  req@ffff9f14ab4ff100 x1862022114864512/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768266 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768241.163380:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff100 x1862022114864512/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768266 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768241.163387:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768241.163390:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768241.163393:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114864512:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768241.163398:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114864640:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768241.163415:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768241.163424:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768241/real 1775768241]  req@ffff9f14ab4fce00 x1862022114864640/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768266 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768241.163430:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fce00 x1862022114864640/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768266 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768241.163435:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768241.163437:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768241.163438:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114864640:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768241.163441:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114864768:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768241.163451:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768241.163456:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768241/real 1775768241]  req@ffff9f14ab4fdc00 x1862022114864768/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768266 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768241.163469:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdc00 x1862022114864768/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768266 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768241.163482:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768241.163485:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768241.163488:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114864768:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775768241.418710:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768242.186652:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768242.442770:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768243.210669:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775768243.466706:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768244.234833:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775768244.490715:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768245.258657:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768245.514731:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768246.282645:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:2.0:1775768246.282674:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768246.282678:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1775768246.282681:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768246.282688:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5623
00000100:00080000:2.0:1775768246.282696:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768246.282701:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775768246.282731:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2d80 x1862022114864896/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768246.282761:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768246.282764:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775768246.282766:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768246.282768:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5623
00000100:00080000:2.0:1775768246.282771:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768246.282773:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768246.282781:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775768246.282790:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0700 x1862022114865024/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768246.282796:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768246.282799:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775768246.282800:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768246.282802:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5623
00000100:00080000:2.0:1775768246.282803:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768246.282807:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775768246.282809:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114864896:192.168.202.110@tcp:250:kworker.0
00000008:00100000:2.0:1775768246.282810:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775768246.282818:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0380 x1862022114865152/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768246.282823:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768246.282825:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775768246.282826:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768246.282828:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5623
00000100:00080000:2.0:1775768246.282830:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768246.282832:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768246.282834:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775768246.282840:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3b80 x1862022114865280/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768246.282846:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768246.282849:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1775768246.282850:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768246.282853:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5623
00000100:00080000:2.0:1775768246.282858:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768246.282861:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768246.282865:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:0.0:1775768246.282867:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775768246.282879:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3800 x1862022114865408/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768246.282888:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768246.282892:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775768246.282893:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768246.282897:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5623
00000100:00080000:2.0:1775768246.282901:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768246.282907:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775768246.282908:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768246/real 1775768246]  req@ffff9f14852d2d80 x1862022114864896/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768271 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:2.0:1775768246.282912:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775768246.282927:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2680 x1862022114865536/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775768246.282933:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2d80 x1862022114864896/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768271 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775768246.282942:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775768246.282946:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775768246.282946:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775768246.282950:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775768246.282953:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:2.0:1775768246.282957:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5623
00000100:00100000:0.0:1775768246.282958:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114864896:192.168.202.110@tcp:250:kworker.0
00000100:00080000:2.0:1775768246.282963:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768246.282967:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768246.282971:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775768246.282974:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114865024:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775768246.282979:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2a00 x1862022114865664/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768246.282986:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768246.282988:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775768246.282989:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768246.282992:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5623
00000100:00080000:2.0:1775768246.282994:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000800:00000100:0.0:1775768246.282997:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:2.0:1775768246.282999:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768246.283003:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775768246.283012:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1500 x1862022114865792/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775768246.283015:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768246/real 1775768246]  req@ffff9f14852d0700 x1862022114865024/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768271 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768246.283029:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0700 x1862022114865024/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768271 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768246.283035:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768246.283037:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768246.283040:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114865024:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768246.283060:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114865152:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768246.283077:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768246.283086:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768246/real 1775768246]  req@ffff9f14852d0380 x1862022114865152/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768271 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768246.283098:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0380 x1862022114865152/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768271 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768246.283105:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768246.283108:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768246.283112:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114865152:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768246.283118:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114865280:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768246.283137:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768246.283148:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768246/real 1775768246]  req@ffff9f14852d3b80 x1862022114865280/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768271 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768246.283159:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3b80 x1862022114865280/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768271 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768246.283166:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768246.283169:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768246.283172:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114865280:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775768246.283177:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114865408:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768246.283190:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768246.283197:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768246/real 1775768246]  req@ffff9f14852d3800 x1862022114865408/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768271 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768246.283204:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3800 x1862022114865408/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768271 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768246.283208:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768246.283210:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768246.283212:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114865408:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775768246.283214:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114865536:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768246.283223:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768246.283227:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768246/real 1775768246]  req@ffff9f14852d2680 x1862022114865536/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768271 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768246.283234:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2680 x1862022114865536/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768271 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768246.283238:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768246.283239:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768246.283241:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114865536:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768246.283251:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114865664:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768246.283264:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768246.283269:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768246/real 1775768246]  req@ffff9f14852d2a00 x1862022114865664/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768271 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768246.283275:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2a00 x1862022114865664/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768271 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768246.283284:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768246.283285:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768246.283287:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114865664:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768246.283290:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114865792:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768246.283298:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768246.283303:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768246/real 1775768246]  req@ffff9f14852d1500 x1862022114865792/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768271 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768246.283308:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1500 x1862022114865792/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768271 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768246.283313:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768246.283314:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768246.283316:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114865792:192.168.202.110@tcp:8:kworker.0
00010000:00010000:3.0:1775768246.538770:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768247.306678:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768247.562742:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768248.330681:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775768248.586754:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768249.354661:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768249.610752:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775768250.378635:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775768250.634693:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775768251.402639:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:1.0:1775768251.402652:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775768251.402657:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1775768251.402661:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775768251.402669:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5628
00000100:00080000:1.0:1775768251.402678:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775768251.402683:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775768251.402727:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fd880 x1862022114865920/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775768251.402746:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775768251.402750:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775768251.402751:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775768251.402754:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5628
00000100:00080000:1.0:1775768251.402756:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775768251.402759:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775768251.402767:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775768251.402775:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fe300 x1862022114866048/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775768251.402780:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114865920:192.168.202.110@tcp:250:kworker.0
00000100:00080000:1.0:1775768251.402785:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775768251.402788:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775768251.402789:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775768251.402792:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5628
00000100:00080000:1.0:1775768251.402793:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775768251.402795:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775768251.402798:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775768251.402806:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdf80 x1862022114866176/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775768251.402811:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775768251.402814:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775768251.402815:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775768251.402818:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5628
00000100:00080000:1.0:1775768251.402829:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775768251.402831:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775768251.402835:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:0.0:1775768251.402840:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775768251.402848:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fea00 x1862022114866304/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775768251.402858:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775768251.402861:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1775768251.402863:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775768251.402867:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5628
00000100:00080000:1.0:1775768251.402872:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1775768251.402877:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768251/real 1775768251]  req@ffff9f14ab4fd880 x1862022114865920/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768276 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775768251.402878:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775768251.402885:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775768251.402897:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdc00 x1862022114866432/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775768251.402903:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fd880 x1862022114865920/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768276 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775768251.402906:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775768251.402912:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775768251.402915:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775768251.402923:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5628
00000100:00080000:0.0:1775768251.402925:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775768251.402926:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775768251.402932:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00080000:0.0:1775768251.402934:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000008:00100000:1.0:1775768251.402938:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775768251.402939:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114865920:192.168.202.110@tcp:250:kworker.0
00000100:00100000:1.0:1775768251.402952:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fce00 x1862022114866560/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775768251.402961:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775768251.402964:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775768251.402966:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775768251.402971:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5628
00000100:00100000:0.0:1775768251.402973:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114866048:192.168.202.110@tcp:38:kworker.0
00000100:00080000:1.0:1775768251.402976:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775768251.402983:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775768251.402988:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775768251.403001:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff100 x1862022114866688/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1775768251.403002:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:1.0:1775768251.403011:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775768251.403014:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775768251.403015:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775768251.403018:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768251/real 1775768251]  req@ffff9f14ab4fe300 x1862022114866048/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768276 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775768251.403019:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5628
00000100:00080000:1.0:1775768251.403025:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775768251.403030:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775768251.403037:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775768251.403042:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fe300 x1862022114866048/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768276 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775768251.403046:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fc380 x1862022114866816/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775768251.403066:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768251.403071:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768251.403075:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114866048:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768251.403082:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114866176:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768251.403100:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768251.403109:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768251/real 1775768251]  req@ffff9f14ab4fdf80 x1862022114866176/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768276 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768251.403119:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdf80 x1862022114866176/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768276 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768251.403126:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768251.403136:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768251.403140:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114866176:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768251.403145:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114866304:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768251.403163:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768251.403173:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768251/real 1775768251]  req@ffff9f14ab4fea00 x1862022114866304/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768276 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768251.403184:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fea00 x1862022114866304/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768276 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768251.403190:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768251.403193:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768251.403196:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114866304:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775768251.403201:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114866432:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768251.403215:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768251.403223:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768251/real 1775768251]  req@ffff9f14ab4fdc00 x1862022114866432/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768276 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768251.403232:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdc00 x1862022114866432/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768276 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768251.403238:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768251.403241:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768251.403243:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114866432:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775768251.403248:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114866560:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768251.403262:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768251.403270:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768251/real 1775768251]  req@ffff9f14ab4fce00 x1862022114866560/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768276 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768251.403294:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fce00 x1862022114866560/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768276 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768251.403300:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768251.403303:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768251.403306:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114866560:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768251.403321:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114866688:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768251.403338:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768251.403347:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768251/real 1775768251]  req@ffff9f14ab4ff100 x1862022114866688/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768276 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768251.403358:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff100 x1862022114866688/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768276 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768251.403364:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768251.403366:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768251.403369:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114866688:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768251.403374:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114866816:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768251.403387:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768251.403395:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768251/real 1775768251]  req@ffff9f14ab4fc380 x1862022114866816/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768276 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768251.403404:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fc380 x1862022114866816/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768276 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768251.403415:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768251.403418:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768251.403420:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114866816:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775768251.658731:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775768252.426719:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768252.682693:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775768253.450643:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768253.706759:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775768254.474657:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768254.730735:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775768255.498625:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775768255.754805:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1775768256.522706:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1775768256.522710:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:1.0:1775768256.522717:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1775768256.522721:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775768256.522733:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5634
00000100:00080000:1.0:1775768256.522746:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775768256.522754:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775768256.522795:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff800 x1862022114866944/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775768256.522825:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775768256.522829:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775768256.522831:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775768256.522840:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5634
00000100:00080000:1.0:1775768256.522845:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775768256.522850:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775768256.522856:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114866944:192.168.202.110@tcp:250:kworker.0
00000008:00100000:1.0:1775768256.522859:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775768256.522874:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fed80 x1862022114867072/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775768256.522883:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775768256.522886:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775768256.522888:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775768256.522891:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5634
00000100:00080000:1.0:1775768256.522894:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775768256.522897:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775768256.522902:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775768256.522910:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ffb80 x1862022114867200/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775768256.522917:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775768256.522920:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775768256.522921:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775768256.522924:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5634
00000100:00080000:1.0:1775768256.522928:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775768256.522931:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000800:00000100:0.0:1775768256.522932:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000008:00100000:1.0:1775768256.522936:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775768256.522943:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fca80 x1862022114867328/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775768256.522950:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775768256.522953:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1775768256.522955:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775768256.522958:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5634
00000100:00080000:1.0:1775768256.522961:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775768256.522964:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775768256.522968:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775768256.522977:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fc380 x1862022114867456/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775768256.522981:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768256/real 1775768256]  req@ffff9f14ab4ff800 x1862022114866944/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768281 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775768256.522985:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775768256.522989:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775768256.522990:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775768256.522993:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5634
00000100:00080000:1.0:1775768256.522996:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1775768256.522998:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff800 x1862022114866944/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768281 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775768256.523000:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775768256.523004:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00080000:0.0:1775768256.523008:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00100000:1.0:1775768256.523011:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff100 x1862022114867584/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775768256.523012:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768256.523017:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114866944:192.168.202.110@tcp:250:kworker.0
00000100:00080000:1.0:1775768256.523025:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775768256.523027:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775768256.523029:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775768256.523033:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5634
00000100:00080000:1.0:1775768256.523036:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1775768256.523039:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114867072:192.168.202.110@tcp:38:kworker.0
00000100:00080000:1.0:1775768256.523039:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775768256.523043:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775768256.523051:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fce00 x1862022114867712/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1775768256.523059:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:1.0:1775768256.523059:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775768256.523063:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775768256.523064:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775768256.523067:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5634
00000100:00100000:0.0:1775768256.523069:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768256/real 1775768256]  req@ffff9f14ab4fed80 x1862022114867072/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768281 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775768256.523070:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775768256.523074:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775768256.523078:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775768256.523079:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fed80 x1862022114867072/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768281 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775768256.523086:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdc00 x1862022114867840/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775768256.523088:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768256.523091:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768256.523094:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114867072:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768256.523099:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ffb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114867200:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768256.523117:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768256.523126:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768256/real 1775768256]  req@ffff9f14ab4ffb80 x1862022114867200/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768281 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768256.523137:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ffb80 x1862022114867200/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768281 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768256.523146:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768256.523148:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768256.523151:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ffb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114867200:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768256.523155:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114867328:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768256.523174:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768256.523181:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768256/real 1775768256]  req@ffff9f14ab4fca80 x1862022114867328/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768281 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768256.523188:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fca80 x1862022114867328/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768281 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768256.523193:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768256.523195:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768256.523197:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114867328:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775768256.523200:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114867456:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768256.523210:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768256.523214:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768256/real 1775768256]  req@ffff9f14ab4fc380 x1862022114867456/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768281 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768256.523222:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fc380 x1862022114867456/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768281 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768256.523229:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768256.523231:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768256.523234:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114867456:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775768256.523239:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114867584:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768256.523257:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768256.523267:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768256/real 1775768256]  req@ffff9f14ab4ff100 x1862022114867584/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768281 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768256.523278:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff100 x1862022114867584/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768281 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768256.523295:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768256.523298:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768256.523301:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114867584:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768256.523317:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114867712:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768256.523331:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768256.523338:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768256/real 1775768256]  req@ffff9f14ab4fce00 x1862022114867712/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768281 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768256.523345:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fce00 x1862022114867712/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768281 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768256.523349:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768256.523351:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768256.523353:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114867712:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768256.523356:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114867840:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768256.523373:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768256.523378:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768256/real 1775768256]  req@ffff9f14ab4fdc00 x1862022114867840/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768281 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768256.523385:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdc00 x1862022114867840/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768281 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768256.523390:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768256.523392:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768256.523394:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114867840:192.168.202.110@tcp:8:kworker.0
00010000:00010000:3.0:1775768256.778785:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775768257.546678:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775768257.802727:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775768258.570657:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775768258.826717:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775768259.594699:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768259.850736:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775768260.618667:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768260.874793:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1775768261.642706:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768261.642715:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1775768261.642719:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768261.642728:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5639
00000100:00080000:2.0:1775768261.642740:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768261.642747:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775768261.642786:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1f80 x1862022114867968/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768261.642814:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768261.642817:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775768261.642818:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768261.642823:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5639
00000100:00080000:2.0:1775768261.642825:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768261.642829:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768261.642837:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775768261.642847:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0a80 x1862022114868096/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775768261.642855:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114867968:192.168.202.110@tcp:250:kworker.0
00000100:00080000:2.0:1775768261.642857:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768261.642862:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775768261.642864:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:1.0:1775768261.642866:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:2.0:1775768261.642867:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5639
00000100:00080000:2.0:1775768261.642870:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768261.642873:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768261.642877:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775768261.642886:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1500 x1862022114868224/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768261.642894:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768261.642897:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775768261.642898:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768261.642900:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5639
00000100:00080000:2.0:1775768261.642902:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768261.642904:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768261.642907:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775768261.642913:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2a00 x1862022114868352/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768261.642918:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768261.642921:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1775768261.642922:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768261.642924:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5639
00000100:00080000:2.0:1775768261.642926:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768261.642928:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768261.642931:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775768261.642937:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2680 x1862022114868480/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768261.642942:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768261.642944:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775768261.642945:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768261.642947:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5639
00000100:00080000:2.0:1775768261.642950:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768261.642953:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768261.642957:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775768261.642964:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3800 x1862022114868608/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768261.642970:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1775768261.642973:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114868096:192.168.202.110@tcp:38:kworker.0
00000100:00080000:2.0:1775768261.642974:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775768261.642977:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768261.642984:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5639
00000100:00080000:2.0:1775768261.642987:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768261.642989:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768261.642992:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775768261.643001:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3b80 x1862022114868736/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768261.643006:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768261.643009:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775768261.643010:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768261.643015:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5639
00000100:00080000:2.0:1775768261.643018:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1775768261.643020:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114868224:192.168.202.110@tcp:38:kworker.0
00000100:00080000:2.0:1775768261.643020:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768261.643022:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775768261.643032:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0380 x1862022114868864/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775768261.643042:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114868352:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775768261.643063:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114868480:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775768261.643090:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114868608:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768261.643113:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114868736:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768261.643131:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114868864:192.168.202.110@tcp:8:kworker.0
00010000:00010000:3.0:1775768261.898739:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775768262.666762:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775768262.922756:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775768263.690713:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768263.946721:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775768264.714838:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000400:00000100:2.0:1775768264.714860:0:2391:0:(lib-socket.c:454:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.202.110:988
00000400:00000100:2.0:1775768264.714925:0:2391:0:(acceptor.c:111:lnet_connect_console_error()) Connection to 192.168.202.110@tcp at host 192.168.202.110 was unreachable: the network or that node may be down, or Lustre may be misconfigured.
00000800:00000100:2.0:1775768264.714946:0:2391:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:2.0:1775768264.714988:0:2391:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:2.0:1775768264.715000:0:2391:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:2.0:1775768264.715009:0:2391:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:2.0:1775768264.715016:0:2391:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:2.0:1775768264.715024:0:2391:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000100:00100000:0.0:1775768264.715026:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768261/real 1775768264]  req@ffff9f14852d0a80 x1862022114868096/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768286 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000800:00000100:2.0:1775768264.715031:0:2391:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:2.0:1775768264.715041:0:2391:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000100:00100000:0.0:1775768264.715068:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0a80 x1862022114868096/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768286 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768264.715082:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768264.715089:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768264.715093:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114868096:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768264.715101:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768261/real 1775768264]  req@ffff9f14852d1500 x1862022114868224/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768286 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768264.715115:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1500 x1862022114868224/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768286 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768264.715121:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768264.715126:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768264.715131:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114868224:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768264.715146:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768261/real 1775768264]  req@ffff9f14852d2a00 x1862022114868352/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768286 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768264.715160:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2a00 x1862022114868352/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768286 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768264.715168:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768264.715174:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768264.715177:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114868352:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775768264.715185:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768261/real 1775768264]  req@ffff9f14852d2680 x1862022114868480/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768286 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768264.715197:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2680 x1862022114868480/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768286 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768264.715204:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768264.715208:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768264.715213:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114868480:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775768264.715221:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768261/real 1775768264]  req@ffff9f14852d1f80 x1862022114867968/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768286 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768264.715256:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1f80 x1862022114867968/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768286 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768264.715264:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768264.715267:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768264.715271:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114867968:192.168.202.110@tcp:250:kworker.0
00000100:00100000:0.0:1775768264.715302:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768261/real 1775768264]  req@ffff9f14852d3800 x1862022114868608/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768286 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768264.715313:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3800 x1862022114868608/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768286 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768264.715321:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768264.715324:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768264.715327:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114868608:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768264.715334:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768261/real 1775768264]  req@ffff9f14852d3b80 x1862022114868736/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768286 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768264.715344:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3b80 x1862022114868736/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768286 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768264.715350:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768264.715353:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768264.715356:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114868736:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768264.715363:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768261/real 1775768264]  req@ffff9f14852d0380 x1862022114868864/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768286 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768264.715380:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0380 x1862022114868864/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768286 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768264.715387:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768264.715390:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768264.715394:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114868864:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775768264.970716:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775768265.738622:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775768265.994700:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1775768266.762642:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775768266.762649:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775768266.762653:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00100000:2.0:1775768266.762661:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775768266.762666:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5644
00000100:00080000:3.0:1775768266.762675:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768266.762680:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775768266.762709:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745500 x1862022114868992/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775768266.762728:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775768266.762731:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775768266.762733:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768266.762736:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5644
00000100:00080000:3.0:1775768266.762740:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768266.762743:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775768266.762752:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775768266.762763:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746680 x1862022114869120/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775768266.762772:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775768266.762775:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775768266.762776:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768266.762779:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5644
00000100:00100000:0.0:1775768266.762781:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114868992:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775768266.762782:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768266.762785:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775768266.762790:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775768266.762797:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745c00 x1862022114869248/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775768266.762805:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775768266.762807:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775768266.762808:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768266.762811:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5644
00000100:00080000:3.0:1775768266.762814:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768266.762816:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775768266.762820:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775768266.762827:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745180 x1862022114869376/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775768266.762834:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775768266.762837:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775768266.762838:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000800:00000100:0.0:1775768266.762840:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775768266.762841:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5644
00000100:00080000:3.0:1775768266.762844:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768266.762847:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775768266.762851:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775768266.762858:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747100 x1862022114869504/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775768266.762879:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775768266.762884:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775768266.762885:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775768266.762887:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768266/real 1775768266]  req@ffff9f1485745500 x1862022114868992/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768291 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775768266.762889:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5644
00000100:00080000:3.0:1775768266.762893:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768266.762896:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775768266.762901:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775768266.762903:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745500 x1862022114868992/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768291 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775768266.762909:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744000 x1862022114869632/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775768266.762912:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768266.762916:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:3.0:1775768266.762917:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1775768266.762920:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114868992:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775768266.762920:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775768266.762921:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768266.762924:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5644
00000100:00100000:0.0:1775768266.762927:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114869120:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775768266.762928:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768266.762930:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775768266.762935:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775768266.762944:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747800 x1862022114869760/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1775768266.762946:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775768266.762953:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1775768266.762956:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768266/real 1775768266]  req@ffff9f1485746680 x1862022114869120/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768291 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775768266.762956:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775768266.762957:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768266.762961:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5644
00000100:00080000:3.0:1775768266.762964:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1775768266.762967:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746680 x1862022114869120/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768291 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775768266.762968:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775768266.762972:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00080000:0.0:1775768266.762976:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768266.762979:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775768266.762979:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745f80 x1862022114869888/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775768266.762982:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114869120:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768266.763005:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114869248:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768266.763024:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768266.763036:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768266/real 1775768266]  req@ffff9f1485745c00 x1862022114869248/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768291 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768266.763047:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745c00 x1862022114869248/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768291 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768266.763055:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768266.763058:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768266.763061:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114869248:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768266.763066:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114869376:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768266.763086:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768266.763095:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768266/real 1775768266]  req@ffff9f1485745180 x1862022114869376/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768291 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768266.763107:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745180 x1862022114869376/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768291 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768266.763114:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768266.763118:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768266.763121:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114869376:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775768266.763127:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114869504:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768266.763144:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768266.763155:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768266/real 1775768266]  req@ffff9f1485747100 x1862022114869504/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768291 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768266.763163:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747100 x1862022114869504/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768291 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768266.763170:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768266.763173:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768266.763176:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114869504:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775768266.763181:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114869632:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768266.763197:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768266.763206:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768266/real 1775768266]  req@ffff9f1485744000 x1862022114869632/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768291 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768266.763216:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744000 x1862022114869632/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768291 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768266.763223:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768266.763226:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768266.763228:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114869632:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768266.763233:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114869760:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768266.763250:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768266.763260:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768266/real 1775768266]  req@ffff9f1485747800 x1862022114869760/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768291 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768266.763269:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747800 x1862022114869760/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768291 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768266.763275:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768266.763278:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768266.763280:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114869760:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768266.763286:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114869888:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768266.763302:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768266.763310:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768266/real 1775768266]  req@ffff9f1485745f80 x1862022114869888/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768291 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768266.763319:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745f80 x1862022114869888/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768291 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768266.763325:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768266.763328:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768266.763330:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114869888:192.168.202.110@tcp:8:kworker.0
00010000:00010000:3.0:1775768267.018708:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768267.786627:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768268.042700:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768268.810644:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768269.066715:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768269.834682:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775768270.090739:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768270.858629:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768271.114746:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1775768271.882657:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:2.0:1775768271.882664:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775768271.882667:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775768271.882672:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768271.882681:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5649
00000100:00080000:3.0:1775768271.882693:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768271.882699:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775768271.882736:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744700 x1862022114870016/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775768271.882762:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775768271.882766:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775768271.882767:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768271.882771:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5649
00000100:00080000:3.0:1775768271.882775:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768271.882779:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775768271.882787:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775768271.882799:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114870016:192.168.202.110@tcp:250:kworker.0
00000100:00100000:3.0:1775768271.882803:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745f80 x1862022114870144/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775768271.882813:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775768271.882817:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775768271.882818:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768271.882822:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5649
00000100:00080000:3.0:1775768271.882825:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768271.882829:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775768271.882834:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775768271.882842:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747800 x1862022114870272/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775768271.882850:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000800:00000100:0.0:1775768271.882854:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775768271.882856:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775768271.882859:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768271.882863:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5649
00000100:00080000:3.0:1775768271.882867:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768271.882870:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775768271.882876:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775768271.882888:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768271/real 1775768271]  req@ffff9f1485744700 x1862022114870016/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768296 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775768271.882889:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744000 x1862022114870400/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775768271.882910:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1775768271.882913:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744700 x1862022114870016/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768296 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775768271.882916:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775768271.882920:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775768271.882927:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775768271.882927:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5649
00000100:00080000:3.0:1775768271.882933:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775768271.882934:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768271.882941:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114870016:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775768271.882941:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775768271.882949:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775768271.882960:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747100 x1862022114870528/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775768271.882962:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114870144:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775768271.882975:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775768271.882979:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775768271.882981:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000800:00000100:0.0:1775768271.882983:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775768271.882988:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5649
00000100:00080000:3.0:1775768271.882993:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1775768271.882996:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768271/real 1775768271]  req@ffff9f1485745f80 x1862022114870144/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768296 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775768271.883000:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775768271.883009:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775768271.883013:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745f80 x1862022114870144/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768296 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768271.883019:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00100000:3.0:1775768271.883023:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745180 x1862022114870656/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775768271.883025:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768271.883030:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114870144:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768271.883039:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114870272:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775768271.883045:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775768271.883048:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775768271.883050:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000800:00000100:0.0:1775768271.883054:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775768271.883056:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5649
00000100:00080000:3.0:1775768271.883061:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768271.883065:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775768271.883070:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775768271.883072:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768271/real 1775768271]  req@ffff9f1485747800 x1862022114870272/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768296 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775768271.883085:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745c00 x1862022114870784/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775768271.883086:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747800 x1862022114870272/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768296 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768271.883103:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775768271.883103:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775768271.883109:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00080000:3.0:1775768271.883110:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00100000:0.0:1775768271.883113:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114870272:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775768271.883114:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775768271.883122:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114870400:192.168.202.110@tcp:8:kworker.0
00000100:00080000:3.0:1775768271.883124:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5649
00000100:00080000:3.0:1775768271.883129:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768271.883134:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000800:00000100:0.0:1775768271.883139:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000008:00100000:3.0:1775768271.883139:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775768271.883146:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768271/real 1775768271]  req@ffff9f1485744000 x1862022114870400/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768296 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775768271.883153:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746680 x1862022114870912/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775768271.883155:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744000 x1862022114870400/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768296 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768271.883182:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768271.883186:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768271.883190:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114870400:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775768271.883208:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114870528:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768271.883228:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768271.883237:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768271/real 1775768271]  req@ffff9f1485747100 x1862022114870528/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768296 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768271.883245:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747100 x1862022114870528/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768296 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768271.883250:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768271.883252:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768271.883256:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114870528:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775768271.883260:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114870656:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768271.883274:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768271.883287:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768271/real 1775768271]  req@ffff9f1485745180 x1862022114870656/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768296 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768271.883295:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745180 x1862022114870656/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768296 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768271.883301:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768271.883304:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768271.883308:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114870656:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768271.883314:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114870784:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768271.883338:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768271.883352:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768271/real 1775768271]  req@ffff9f1485745c00 x1862022114870784/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768296 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768271.883362:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745c00 x1862022114870784/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768296 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768271.883367:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768271.883371:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768271.883373:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114870784:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768271.883377:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114870912:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768271.883393:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768271.883404:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768271/real 1775768271]  req@ffff9f1485746680 x1862022114870912/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768296 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768271.883417:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746680 x1862022114870912/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768296 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768271.883425:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768271.883429:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768271.883434:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114870912:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775768272.138722:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768272.906640:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775768273.162707:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768273.930671:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768274.186693:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768274.954631:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768275.210708:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768275.978675:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768276.234716:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1775768277.002615:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:2.0:1775768277.002623:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775768277.002624:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775768277.002629:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768277.002640:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5654
00000100:00080000:3.0:1775768277.002650:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768277.002657:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775768277.002692:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745500 x1862022114871040/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775768277.002723:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775768277.002726:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775768277.002727:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768277.002731:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5654
00000100:00080000:3.0:1775768277.002734:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768277.002738:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775768277.002748:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775768277.002759:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744e00 x1862022114871168/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775768277.002766:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114871040:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775768277.002772:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775768277.002775:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775768277.002777:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768277.002781:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5654
00000100:00080000:3.0:1775768277.002785:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768277.002788:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775768277.002792:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775768277.002803:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744380 x1862022114871296/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775768277.002811:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775768277.002814:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775768277.002815:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768277.002819:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5654
00000100:00080000:3.0:1775768277.002822:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768277.002825:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000800:00000100:0.0:1775768277.002830:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000008:00100000:3.0:1775768277.002831:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775768277.002843:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746a00 x1862022114871424/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775768277.002851:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775768277.002855:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775768277.002856:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768277.002860:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5654
00000100:00080000:3.0:1775768277.002864:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1775768277.002869:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768277/real 1775768277]  req@ffff9f1485745500 x1862022114871040/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768302 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775768277.002869:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775768277.002873:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775768277.002887:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744a80 x1862022114871552/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775768277.002894:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745500 x1862022114871040/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768302 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775768277.002899:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775768277.002906:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775768277.002910:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768277.002915:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5654
00000100:00080000:3.0:1775768277.002918:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775768277.002921:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775768277.002922:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00080000:0.0:1775768277.002931:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000008:00100000:3.0:1775768277.002931:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775768277.002937:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114871040:192.168.202.110@tcp:250:kworker.0
00000100:00100000:3.0:1775768277.002946:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747480 x1862022114871680/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775768277.002956:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775768277.002959:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775768277.002961:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775768277.002963:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114871168:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775768277.002967:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5654
00000100:00080000:3.0:1775768277.002972:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768277.002975:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775768277.002980:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775768277.002992:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745880 x1862022114871808/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1775768277.002994:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775768277.003006:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1775768277.003010:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768277/real 1775768277]  req@ffff9f1485744e00 x1862022114871168/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768302 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775768277.003010:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775768277.003013:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768277.003022:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5654
00000100:00080000:3.0:1775768277.003028:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1775768277.003032:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744e00 x1862022114871168/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768302 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775768277.003033:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775768277.003041:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00080000:0.0:1775768277.003044:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768277.003049:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775768277.003054:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746300 x1862022114871936/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775768277.003055:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114871168:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768277.003066:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114871296:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768277.003087:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768277.003095:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768277/real 1775768277]  req@ffff9f1485744380 x1862022114871296/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768302 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768277.003106:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744380 x1862022114871296/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768302 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768277.003116:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768277.003118:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768277.003119:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114871296:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768277.003123:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114871424:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768277.003155:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768277.003166:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768277/real 1775768277]  req@ffff9f1485746a00 x1862022114871424/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768302 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768277.003177:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746a00 x1862022114871424/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768302 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768277.003185:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768277.003188:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768277.003191:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114871424:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775768277.003196:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114871552:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768277.003213:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768277.003222:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768277/real 1775768277]  req@ffff9f1485744a80 x1862022114871552/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768302 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768277.003232:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744a80 x1862022114871552/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768302 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768277.003239:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768277.003242:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768277.003245:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114871552:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775768277.003251:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114871680:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768277.003268:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768277.003286:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768277/real 1775768277]  req@ffff9f1485747480 x1862022114871680/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768302 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768277.003297:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747480 x1862022114871680/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768302 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768277.003304:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768277.003308:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768277.003311:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114871680:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768277.003327:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114871808:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768277.003345:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768277.003354:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768277/real 1775768277]  req@ffff9f1485745880 x1862022114871808/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768302 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768277.003364:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745880 x1862022114871808/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768302 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768277.003371:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768277.003374:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768277.003377:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114871808:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768277.003383:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114871936:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768277.003398:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768277.003408:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768277/real 1775768277]  req@ffff9f1485746300 x1862022114871936/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768302 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768277.003418:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746300 x1862022114871936/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768302 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768277.003425:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768277.003428:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768277.003431:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114871936:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775768277.258700:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768278.026619:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768278.282797:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775768279.050690:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775768279.306670:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775768280.074659:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775768280.330733:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775768281.098683:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775768281.354752:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775768282.122678:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:2.0:1775768282.122680:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768282.122688:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1775768282.122691:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768282.122700:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5659
00000100:00080000:2.0:1775768282.122709:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768282.122715:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775768282.122748:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0700 x1862022114872064/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768282.122769:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768282.122771:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775768282.122772:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768282.122775:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5659
00000100:00080000:2.0:1775768282.122777:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768282.122779:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768282.122786:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775768282.122793:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2d80 x1862022114872192/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768282.122800:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768282.122802:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775768282.122804:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768282.122806:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5659
00000100:00080000:2.0:1775768282.122807:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768282.122811:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775768282.122812:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114872064:192.168.202.110@tcp:250:kworker.0
00000008:00100000:2.0:1775768282.122815:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775768282.122826:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0000 x1862022114872320/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768282.122835:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768282.122838:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775768282.122839:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768282.122842:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5659
00000100:00080000:2.0:1775768282.122846:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768282.122849:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768282.122853:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775768282.122864:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1180 x1862022114872448/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768282.122874:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768282.122877:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1775768282.122878:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000800:00000100:0.0:1775768282.122885:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:2.0:1775768282.122889:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5659
00000100:00080000:2.0:1775768282.122894:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768282.122898:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768282.122903:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775768282.122914:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1880 x1862022114872576/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768282.122923:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1775768282.122926:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768282/real 1775768282]  req@ffff9f14852d0700 x1862022114872064/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768307 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775768282.122927:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775768282.122930:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768282.122937:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5659
00000100:00080000:2.0:1775768282.122943:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768282.122948:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768282.122952:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775768282.122960:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0e00 x1862022114872704/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775768282.122964:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0700 x1862022114872064/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768307 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775768282.122969:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768282.122975:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775768282.122978:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768282.122984:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5659
00000100:00080000:2.0:1775768282.122986:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775768282.122987:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775768282.122988:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768282.122994:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00080000:0.0:1775768282.122997:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775768282.123001:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2300 x1862022114872832/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775768282.123004:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114872064:192.168.202.110@tcp:250:kworker.0
00000100:00080000:2.0:1775768282.123012:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768282.123015:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775768282.123017:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775768282.123018:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114872192:192.168.202.110@tcp:38:kworker.0
00000100:00080000:2.0:1775768282.123021:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5659
00000100:00080000:2.0:1775768282.123026:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768282.123028:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768282.123031:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775768282.123038:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3100 x1862022114872960/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1775768282.123050:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768282.123064:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768282/real 1775768282]  req@ffff9f14852d2d80 x1862022114872192/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768307 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768282.123083:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2d80 x1862022114872192/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768307 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768282.123092:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768282.123095:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768282.123098:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114872192:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768282.123119:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114872320:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768282.123137:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768282.123147:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768282/real 1775768282]  req@ffff9f14852d0000 x1862022114872320/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768307 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768282.123159:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0000 x1862022114872320/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768307 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768282.123166:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768282.123169:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768282.123172:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114872320:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768282.123177:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114872448:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768282.123193:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768282.123202:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768282/real 1775768282]  req@ffff9f14852d1180 x1862022114872448/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768307 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768282.123212:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1180 x1862022114872448/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768307 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768282.123225:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768282.123228:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768282.123231:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114872448:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775768282.123236:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114872576:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768282.123252:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768282.123261:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768282/real 1775768282]  req@ffff9f14852d1880 x1862022114872576/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768307 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768282.123280:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1880 x1862022114872576/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768307 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768282.123288:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768282.123291:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768282.123294:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114872576:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775768282.123299:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114872704:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768282.123317:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768282.123326:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768282/real 1775768282]  req@ffff9f14852d0e00 x1862022114872704/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768307 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768282.123336:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0e00 x1862022114872704/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768307 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768282.123343:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768282.123346:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768282.123350:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114872704:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768282.123355:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114872832:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768282.123372:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768282.123381:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768282/real 1775768282]  req@ffff9f14852d2300 x1862022114872832/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768307 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768282.123415:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2300 x1862022114872832/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768307 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768282.123423:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768282.123426:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768282.123429:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114872832:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768282.123434:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114872960:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768282.123460:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768282.123470:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768282/real 1775768282]  req@ffff9f14852d3100 x1862022114872960/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768307 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768282.123480:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3100 x1862022114872960/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768307 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768282.123487:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768282.123490:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768282.123493:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114872960:192.168.202.110@tcp:8:kworker.0
00010000:00010000:3.0:1775768282.378724:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775768283.146696:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775768283.402721:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775768284.170656:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775768284.426788:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768285.194706:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768285.450787:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775768286.219907:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775768286.474812:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1775768287.242655:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:3.0:1775768287.242657:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:2.0:1775768287.242668:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1775768287.242673:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768287.242685:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5664
00000100:00080000:2.0:1775768287.242699:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768287.242708:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775768287.242749:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1c00 x1862022114873088/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768287.242784:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768287.242788:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775768287.242789:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768287.242794:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5664
00000100:00080000:2.0:1775768287.242797:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768287.242801:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768287.242812:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775768287.242825:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3480 x1862022114873216/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768287.242835:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768287.242841:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00100000:0.0:1775768287.242842:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114873088:192.168.202.110@tcp:250:kworker.0
00000100:00080000:2.0:1775768287.242843:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768287.242851:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5664
00000100:00080000:2.0:1775768287.242855:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768287.242858:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768287.242863:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775768287.242877:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3100 x1862022114873344/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768287.242886:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768287.242890:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775768287.242891:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768287.242895:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5664
00000100:00080000:2.0:1775768287.242899:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768287.242903:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768287.242908:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775768287.242922:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2300 x1862022114873472/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1775768287.242923:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:2.0:1775768287.242932:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768287.242935:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1775768287.242936:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768287.242939:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5664
00000100:00080000:2.0:1775768287.242943:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768287.242945:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768287.242951:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775768287.242958:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0e00 x1862022114873600/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768287.242965:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768287.242967:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775768287.242969:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768287.242971:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5664
00000100:00080000:2.0:1775768287.242973:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1775768287.242977:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768287/real 1775768287]  req@ffff9f14852d1c00 x1862022114873088/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768312 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775768287.242978:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768287.242983:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775768287.242992:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1880 x1862022114873728/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768287.243010:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768287.243014:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775768287.243015:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768287.243018:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5664
00000100:00080000:2.0:1775768287.243020:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768287.243022:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768287.243025:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775768287.243028:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1c00 x1862022114873088/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768312 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775768287.243032:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1180 x1862022114873856/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768287.243058:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775768287.243060:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775768287.243062:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775768287.243066:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775768287.243069:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:2.0:1775768287.243074:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5664
00000100:00100000:0.0:1775768287.243077:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114873088:192.168.202.110@tcp:250:kworker.0
00000100:00080000:2.0:1775768287.243079:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768287.243086:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768287.243091:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775768287.243102:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0000 x1862022114873984/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775768287.243117:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114873216:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768287.243145:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768287.243158:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768287/real 1775768287]  req@ffff9f14852d3480 x1862022114873216/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768312 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768287.243170:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3480 x1862022114873216/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768312 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768287.243178:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768287.243181:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768287.243184:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114873216:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768287.243190:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114873344:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768287.243209:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768287.243224:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768287/real 1775768287]  req@ffff9f14852d3100 x1862022114873344/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768312 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768287.243236:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3100 x1862022114873344/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768312 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768287.243250:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768287.243253:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768287.243255:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114873344:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768287.243261:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114873472:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768287.243280:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768287.243288:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768287/real 1775768287]  req@ffff9f14852d2300 x1862022114873472/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768312 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768287.243299:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2300 x1862022114873472/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768312 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768287.243306:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768287.243309:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768287.243312:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114873472:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775768287.243317:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114873600:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768287.243336:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768287.243358:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768287/real 1775768287]  req@ffff9f14852d0e00 x1862022114873600/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768312 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768287.243370:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0e00 x1862022114873600/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768312 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768287.243378:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768287.243380:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768287.243384:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114873600:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775768287.243397:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114873728:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768287.243417:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768287.243427:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768287/real 1775768287]  req@ffff9f14852d1880 x1862022114873728/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768312 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768287.243438:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1880 x1862022114873728/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768312 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768287.243446:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768287.243450:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768287.243453:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114873728:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768287.243459:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114873856:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768287.243478:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768287.243488:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768287/real 1775768287]  req@ffff9f14852d1180 x1862022114873856/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768312 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768287.243499:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1180 x1862022114873856/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768312 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768287.243507:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768287.243510:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768287.243513:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114873856:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768287.243535:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114873984:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768287.243578:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768287.243589:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768287/real 1775768287]  req@ffff9f14852d0000 x1862022114873984/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768312 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768287.243601:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0000 x1862022114873984/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768312 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768287.243609:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768287.243612:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768287.243615:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114873984:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775768287.498778:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775768288.266651:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775768288.522873:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775768289.290667:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768289.546714:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775768290.314613:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775768290.570748:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775768291.338685:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775768291.594733:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775768292.362632:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:2.0:1775768292.362644:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768292.362652:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1775768292.362656:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768292.362668:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5669
00000100:00080000:2.0:1775768292.362680:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768292.362687:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775768292.362726:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2d80 x1862022114874112/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768292.362769:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768292.362772:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775768292.362774:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768292.362778:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5669
00000100:00080000:2.0:1775768292.362781:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768292.362785:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768292.362795:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775768292.362808:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0700 x1862022114874240/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775768292.362819:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114874112:192.168.202.110@tcp:250:kworker.0
00000100:00080000:2.0:1775768292.362821:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768292.362826:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775768292.362828:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768292.362833:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5669
00000100:00080000:2.0:1775768292.362836:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768292.362840:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768292.362845:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775768292.362857:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0380 x1862022114874368/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768292.362866:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768292.362869:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775768292.362871:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768292.362875:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5669
00000100:00080000:2.0:1775768292.362879:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768292.362882:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000800:00000100:0.0:1775768292.362886:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000008:00100000:2.0:1775768292.362889:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775768292.362903:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3b80 x1862022114874496/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768292.362913:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768292.362916:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1775768292.362918:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768292.362922:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5669
00000100:00100000:0.0:1775768292.362925:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768292/real 1775768292]  req@ffff9f14852d2d80 x1862022114874112/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768317 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775768292.362928:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768292.362936:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768292.362942:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775768292.362945:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2d80 x1862022114874112/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768317 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768292.362954:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00100000:2.0:1775768292.362956:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3800 x1862022114874624/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775768292.362959:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768292.362966:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114874112:192.168.202.110@tcp:250:kworker.0
00000100:00080000:2.0:1775768292.362976:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768292.362978:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775768292.362980:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768292.362985:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5669
00000100:00080000:2.0:1775768292.362988:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768292.362993:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775768292.362996:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114874240:192.168.202.110@tcp:38:kworker.0
00000008:00100000:2.0:1775768292.363000:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775768292.363014:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1f80 x1862022114874752/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768292.363024:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000800:00000100:0.0:1775768292.363027:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:2.0:1775768292.363031:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775768292.363034:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768292.363039:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5669
00000100:00080000:2.0:1775768292.363044:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1775768292.363046:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768292/real 1775768292]  req@ffff9f14852d0700 x1862022114874240/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768317 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775768292.363050:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768292.363057:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775768292.363066:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0700 x1862022114874240/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768317 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775768292.363070:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2680 x1862022114874880/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768292.363094:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775768292.363098:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775768292.363099:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775768292.363103:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775768292.363106:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00080000:2.0:1775768292.363110:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5669
00000100:00100000:0.0:1775768292.363114:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114874240:192.168.202.110@tcp:38:kworker.0
00000100:00080000:2.0:1775768292.363116:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768292.363123:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775768292.363126:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114874368:192.168.202.110@tcp:38:kworker.0
00000008:00100000:2.0:1775768292.363130:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775768292.363144:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2a00 x1862022114875008/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1775768292.363154:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768292.363167:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768292/real 1775768292]  req@ffff9f14852d0380 x1862022114874368/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768317 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768292.363175:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0380 x1862022114874368/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768317 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768292.363181:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768292.363183:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768292.363185:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114874368:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768292.363189:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114874496:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768292.363203:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768292.363208:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768292/real 1775768292]  req@ffff9f14852d3b80 x1862022114874496/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768317 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768292.363215:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3b80 x1862022114874496/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768317 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768292.363219:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768292.363221:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768292.363223:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114874496:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775768292.363227:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114874624:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768292.363236:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768292.363240:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768292/real 1775768292]  req@ffff9f14852d3800 x1862022114874624/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768317 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768292.363246:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3800 x1862022114874624/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768317 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768292.363258:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768292.363263:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768292.363265:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114874624:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775768292.363277:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114874752:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768292.363287:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768292.363308:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768292/real 1775768292]  req@ffff9f14852d1f80 x1862022114874752/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768317 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768292.363316:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1f80 x1862022114874752/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768317 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768292.363321:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768292.363323:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768292.363325:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114874752:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768292.363328:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114874880:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768292.363342:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768292.363352:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768292/real 1775768292]  req@ffff9f14852d2680 x1862022114874880/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768317 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768292.363363:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2680 x1862022114874880/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768317 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768292.363370:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768292.363373:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768292.363377:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114874880:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768292.363383:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114875008:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768292.363400:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768292.363407:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768292/real 1775768292]  req@ffff9f14852d2a00 x1862022114875008/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768317 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768292.363414:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2a00 x1862022114875008/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768317 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768292.363419:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768292.363421:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768292.363423:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114875008:192.168.202.110@tcp:8:kworker.0
00010000:00010000:3.0:1775768292.618755:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775768293.386678:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775768293.642784:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775768294.410601:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775768294.666749:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768295.434689:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775768295.690722:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768296.458660:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768296.714673:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1775768297.482659:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:2.0:1775768297.482662:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775768297.482667:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775768297.482671:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768297.482679:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5674
00000100:00080000:3.0:1775768297.482689:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768297.482696:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775768297.482732:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747b80 x1862022114875136/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775768297.482755:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775768297.482757:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775768297.482758:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768297.482761:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5674
00000100:00080000:3.0:1775768297.482764:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768297.482766:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775768297.482774:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775768297.482785:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746300 x1862022114875264/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775768297.482796:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1775768297.482798:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114875136:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775768297.482798:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775768297.482802:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768297.482805:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5674
00000100:00080000:3.0:1775768297.482807:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768297.482809:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775768297.482815:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775768297.482826:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745880 x1862022114875392/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775768297.482834:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775768297.482837:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775768297.482839:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768297.482843:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5674
00000100:00080000:3.0:1775768297.482846:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768297.482850:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775768297.482855:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:0.0:1775768297.482861:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775768297.482867:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747480 x1862022114875520/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775768297.482876:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775768297.482878:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775768297.482879:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768297.482882:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5674
00000100:00080000:3.0:1775768297.482885:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768297.482888:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775768297.482892:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775768297.482894:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768297/real 1775768297]  req@ffff9f1485747b80 x1862022114875136/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768322 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775768297.482901:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744a80 x1862022114875648/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775768297.482908:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1775768297.482910:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747b80 x1862022114875136/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768322 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775768297.482918:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775768297.482919:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775768297.482920:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775768297.482924:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:3.0:1775768297.482925:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5674
00000100:00080000:3.0:1775768297.482929:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1775768297.482930:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114875136:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775768297.482934:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775768297.482938:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775768297.482944:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746a00 x1862022114875776/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775768297.482951:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114875264:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775768297.482951:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775768297.482958:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775768297.482960:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768297.482963:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5674
00000100:00080000:3.0:1775768297.482966:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768297.482968:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775768297.482971:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000800:00000100:0.0:1775768297.482974:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775768297.482979:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744380 x1862022114875904/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775768297.482985:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768297/real 1775768297]  req@ffff9f1485746300 x1862022114875264/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768322 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775768297.482990:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775768297.482996:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775768297.482999:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768297.483003:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5674
00000100:00100000:0.0:1775768297.483006:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746300 x1862022114875264/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768322 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775768297.483007:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768297.483013:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775768297.483019:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00080000:0.0:1775768297.483020:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768297.483025:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775768297.483028:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744e00 x1862022114876032/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775768297.483029:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114875264:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768297.483039:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114875392:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768297.483052:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768297.483059:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768297/real 1775768297]  req@ffff9f1485745880 x1862022114875392/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768322 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768297.483065:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745880 x1862022114875392/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768322 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768297.483069:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768297.483071:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768297.483073:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114875392:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768297.483076:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114875520:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768297.483086:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768297.483090:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768297/real 1775768297]  req@ffff9f1485747480 x1862022114875520/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768322 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768297.483096:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747480 x1862022114875520/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768322 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768297.483101:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768297.483102:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768297.483104:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114875520:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775768297.483107:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114875648:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768297.483114:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768297.483119:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768297/real 1775768297]  req@ffff9f1485744a80 x1862022114875648/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768322 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768297.483125:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744a80 x1862022114875648/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768322 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768297.483128:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768297.483130:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768297.483131:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114875648:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775768297.483140:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114875776:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768297.483147:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768297.483152:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768297/real 1775768297]  req@ffff9f1485746a00 x1862022114875776/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768322 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768297.483158:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746a00 x1862022114875776/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768322 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768297.483163:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768297.483164:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768297.483166:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114875776:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768297.483168:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114875904:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768297.483176:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768297.483180:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768297/real 1775768297]  req@ffff9f1485744380 x1862022114875904/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768322 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768297.483186:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744380 x1862022114875904/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768322 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768297.483190:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768297.483191:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768297.483193:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114875904:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768297.483195:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114876032:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768297.483202:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768297.483207:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768297/real 1775768297]  req@ffff9f1485744e00 x1862022114876032/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768322 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768297.483212:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744e00 x1862022114876032/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768322 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768297.483220:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768297.483222:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768297.483223:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114876032:192.168.202.110@tcp:8:kworker.0
00010000:00010000:1.0:1775768297.738723:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768298.506667:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775768298.762699:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768299.530670:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768299.786720:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775768300.554697:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775768300.810662:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775768301.578659:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775768301.834708:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1775768302.602677:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:3.0:1775768302.602679:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:0.0:1775768302.602688:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1775768302.602692:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775768302.602701:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5680
00000100:00080000:0.0:1775768302.602711:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775768302.602718:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775768302.602752:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d2300 x1862022114876160/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775768302.602779:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775768302.602782:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775768302.602784:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775768302.602788:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5680
00000100:00080000:0.0:1775768302.602791:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775768302.602795:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775768302.602804:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775768302.602810:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114876160:192.168.202.110@tcp:250:kworker.0
00000100:00100000:0.0:1775768302.602817:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3800 x1862022114876288/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775768302.602827:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775768302.602830:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:0.0:1775768302.602833:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775768302.602837:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5680
00000100:00080000:0.0:1775768302.602840:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775768302.602844:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775768302.602848:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775768302.602858:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0000 x1862022114876416/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:2.0:1775768302.602861:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:0.0:1775768302.602870:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775768302.602873:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775768302.602875:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775768302.602879:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5680
00000100:00080000:0.0:1775768302.602882:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775768302.602885:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775768302.602898:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775768302.602900:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768302/real 1775768302]  req@ffff9f14a04d2300 x1862022114876160/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768327 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768302.602912:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0e00 x1862022114876544/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775768302.602920:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d2300 x1862022114876160/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768327 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768302.602922:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775768302.602928:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1775768302.602932:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775768302.602940:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5680
00000100:00080000:0.0:1775768302.602946:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775768302.602950:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00080000:2.0:1775768302.602951:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000008:00100000:0.0:1775768302.602957:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00080000:2.0:1775768302.602958:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775768302.602963:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114876160:192.168.202.110@tcp:250:kworker.0
00000100:00100000:0.0:1775768302.602969:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d1500 x1862022114876672/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775768302.602977:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775768302.602981:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775768302.602982:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:2.0:1775768302.602984:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114876288:192.168.202.110@tcp:38:kworker.0
00000100:00080000:0.0:1775768302.602989:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5680
00000100:00080000:0.0:1775768302.602992:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775768302.602995:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775768302.603000:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000800:00000100:2.0:1775768302.603008:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768302.603011:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d2a00 x1862022114876800/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775768302.603018:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768302/real 1775768302]  req@ffff9f14a04d3800 x1862022114876288/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768327 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768302.603026:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775768302.603029:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:0.0:1775768302.603030:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775768302.603036:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5680
00000100:00080000:0.0:1775768302.603042:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:2.0:1775768302.603046:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3800 x1862022114876288/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768327 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768302.603052:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00080000:2.0:1775768302.603053:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000008:00100000:0.0:1775768302.603058:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00080000:2.0:1775768302.603058:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775768302.603062:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114876288:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775768302.603067:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114876416:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768302.603071:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d2300 x1862022114876928/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775768302.603080:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775768302.603084:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000800:00000100:2.0:1775768302.603086:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:0.0:1775768302.603087:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775768302.603093:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5680
00000100:00080000:0.0:1775768302.603097:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:2.0:1775768302.603099:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768302/real 1775768302]  req@ffff9f14a04d0000 x1862022114876416/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768327 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768302.603102:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775768302.603109:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775768302.603118:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0000 x1862022114876416/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768327 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768302.603121:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0a80 x1862022114877056/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768302.603138:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775768302.603145:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775768302.603148:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114876416:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775768302.603154:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114876544:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775768302.603170:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775768302.603178:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768302/real 1775768302]  req@ffff9f14a04d0e00 x1862022114876544/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768327 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775768302.603189:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0e00 x1862022114876544/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768327 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775768302.603206:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775768302.603210:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775768302.603213:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114876544:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775768302.603219:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114876672:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775768302.603244:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775768302.603253:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768302/real 1775768302]  req@ffff9f14a04d1500 x1862022114876672/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768327 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775768302.603263:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1500 x1862022114876672/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768327 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775768302.603267:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775768302.603269:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775768302.603271:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114876672:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775768302.603282:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114876800:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775768302.603294:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775768302.603300:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768302/real 1775768302]  req@ffff9f14a04d2a00 x1862022114876800/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768327 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775768302.603307:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d2a00 x1862022114876800/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768327 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775768302.603319:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775768302.603320:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775768302.603322:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114876800:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775768302.603325:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114876928:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775768302.603335:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775768302.603340:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768302/real 1775768302]  req@ffff9f14a04d2300 x1862022114876928/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768327 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775768302.603346:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d2300 x1862022114876928/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768327 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775768302.603350:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775768302.603352:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775768302.603353:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114876928:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775768302.603356:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114877056:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775768302.603364:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775768302.603368:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768302/real 1775768302]  req@ffff9f14a04d0a80 x1862022114877056/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768327 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775768302.603374:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0a80 x1862022114877056/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768327 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775768302.603382:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775768302.603384:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775768302.603386:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114877056:192.168.202.110@tcp:8:kworker.0
00010000:00010000:0.0:1775768302.858739:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775768303.626634:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775768303.882703:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775768304.650668:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775768304.906743:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768305.674633:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775768305.930694:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768306.698608:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768306.954683:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768307.722653:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775768307.722661:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775768307.722666:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775768307.722670:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768307.722678:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5685
00000100:00080000:3.0:1775768307.722688:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768307.722695:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775768307.722731:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745500 x1862022114877184/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775768307.722759:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775768307.722762:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775768307.722764:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768307.722769:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5685
00000100:00080000:3.0:1775768307.722773:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768307.722776:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775768307.722785:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775768307.722796:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114877184:192.168.202.110@tcp:250:kworker.0
00000100:00100000:3.0:1775768307.722798:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746680 x1862022114877312/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775768307.722810:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775768307.722812:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775768307.722813:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768307.722816:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5685
00000100:00080000:3.0:1775768307.722818:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768307.722820:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775768307.722822:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775768307.722833:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745c00 x1862022114877440/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775768307.722842:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775768307.722845:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775768307.722847:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768307.722850:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5685
00000100:00080000:3.0:1775768307.722855:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768307.722859:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775768307.722865:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:0.0:1775768307.722869:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775768307.722878:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745180 x1862022114877568/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775768307.722885:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775768307.722887:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775768307.722888:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768307.722890:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5685
00000100:00080000:3.0:1775768307.722893:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768307.722896:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775768307.722900:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775768307.722907:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768307/real 1775768307]  req@ffff9f1485745500 x1862022114877184/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768332 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775768307.722908:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747100 x1862022114877696/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775768307.722925:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775768307.722930:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775768307.722932:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768307.722935:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5685
00000100:00080000:3.0:1775768307.722937:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768307.722939:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775768307.722941:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775768307.722944:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745500 x1862022114877184/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768332 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775768307.722949:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744000 x1862022114877824/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775768307.722968:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775768307.722970:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775768307.722970:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775768307.722972:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768307.722979:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5685
00000100:00080000:0.0:1775768307.722980:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:3.0:1775768307.722981:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768307.722985:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775768307.722988:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114877184:192.168.202.110@tcp:250:kworker.0
00000100:00100000:0.0:1775768307.723011:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114877312:192.168.202.110@tcp:38:kworker.0
00000008:00100000:3.0:1775768307.723040:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000800:00000100:0.0:1775768307.723046:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775768307.723054:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747800 x1862022114877952/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775768307.723062:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768307/real 1775768307]  req@ffff9f1485746680 x1862022114877312/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768332 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775768307.723067:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775768307.723075:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775768307.723079:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775768307.723085:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746680 x1862022114877312/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768332 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775768307.723085:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5685
00000100:00080000:3.0:1775768307.723089:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768307.723094:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775768307.723102:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00080000:0.0:1775768307.723112:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00100000:3.0:1775768307.723116:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745f80 x1862022114878080/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775768307.723120:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768307.723127:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114877312:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768307.723138:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114877440:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768307.723172:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768307.723188:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768307/real 1775768307]  req@ffff9f1485745c00 x1862022114877440/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768332 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768307.723200:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745c00 x1862022114877440/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768332 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768307.723207:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768307.723210:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768307.723213:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114877440:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768307.723218:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114877568:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768307.723234:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768307.723241:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768307/real 1775768307]  req@ffff9f1485745180 x1862022114877568/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768332 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768307.723251:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745180 x1862022114877568/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768332 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768307.723258:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768307.723261:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768307.723265:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114877568:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775768307.723270:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114877696:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768307.723285:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768307.723294:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768307/real 1775768307]  req@ffff9f1485747100 x1862022114877696/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768332 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768307.723304:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747100 x1862022114877696/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768332 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768307.723312:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768307.723315:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768307.723317:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114877696:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775768307.723323:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114877824:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768307.723340:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768307.723351:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768307/real 1775768307]  req@ffff9f1485744000 x1862022114877824/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768332 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768307.723362:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744000 x1862022114877824/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768332 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768307.723369:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768307.723372:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768307.723375:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114877824:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768307.723404:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114877952:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768307.723422:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768307.723434:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768307/real 1775768307]  req@ffff9f1485747800 x1862022114877952/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768332 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768307.723445:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747800 x1862022114877952/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768332 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768307.723454:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768307.723458:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768307.723462:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114877952:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768307.723469:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114878080:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768307.723482:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768307.723490:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768307/real 1775768307]  req@ffff9f1485745f80 x1862022114878080/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768332 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768307.723501:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745f80 x1862022114878080/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768332 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768307.723509:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768307.723513:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768307.723516:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114878080:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775768307.978716:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768308.746666:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775768309.002701:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768309.770651:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:0.0:1775768310.026665:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768310.794658:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775768311.050752:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768311.818655:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775768312.074739:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1775768312.842694:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:2.0:1775768312.842697:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775768312.842708:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775768312.842713:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768312.842725:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5690
00000100:00080000:3.0:1775768312.842739:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768312.842747:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775768312.842788:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744700 x1862022114878208/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775768312.842818:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775768312.842822:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775768312.842824:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768312.842828:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5690
00000100:00080000:3.0:1775768312.842832:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768312.842836:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775768312.842845:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775768312.842859:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745f80 x1862022114878336/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775768312.842868:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775768312.842872:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775768312.842873:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775768312.842876:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114878208:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775768312.842877:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5690
00000100:00080000:3.0:1775768312.842880:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768312.842883:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775768312.842888:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775768312.842897:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747800 x1862022114878464/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775768312.842905:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775768312.842908:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775768312.842909:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768312.842912:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5690
00000100:00080000:3.0:1775768312.842915:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768312.842918:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775768312.842922:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775768312.842930:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744000 x1862022114878592/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775768312.842938:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775768312.842941:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775768312.842942:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000800:00000100:0.0:1775768312.842944:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775768312.842946:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5690
00000100:00080000:3.0:1775768312.842948:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768312.842951:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775768312.842955:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775768312.842962:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747100 x1862022114878720/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775768312.842970:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775768312.842973:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775768312.842974:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768312.842979:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5690
00000100:00080000:3.0:1775768312.842982:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768312.842985:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775768312.842990:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768312/real 1775768312]  req@ffff9f1485744700 x1862022114878208/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768337 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:3.0:1775768312.842990:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775768312.842997:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745180 x1862022114878848/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775768312.843007:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744700 x1862022114878208/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768337 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775768312.843010:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775768312.843013:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775768312.843015:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775768312.843018:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775768312.843018:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5690
00000100:00080000:3.0:1775768312.843021:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775768312.843022:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:3.0:1775768312.843024:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775768312.843026:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114878208:192.168.202.110@tcp:250:kworker.0
00000008:00100000:3.0:1775768312.843028:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775768312.843036:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745c00 x1862022114878976/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775768312.843043:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775768312.843046:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775768312.843047:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768312.843051:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5690
00000100:00080000:3.0:1775768312.843054:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768312.843057:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775768312.843061:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775768312.843064:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114878336:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775768312.843069:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746680 x1862022114879104/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1775768312.843085:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768312.843114:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768312/real 1775768312]  req@ffff9f1485745f80 x1862022114878336/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768337 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768312.843126:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745f80 x1862022114878336/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768337 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768312.843134:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768312.843137:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768312.843140:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114878336:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768312.843146:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114878464:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768312.843161:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768312.843168:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768312/real 1775768312]  req@ffff9f1485747800 x1862022114878464/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768337 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768312.843175:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747800 x1862022114878464/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768337 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768312.843180:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768312.843182:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768312.843184:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114878464:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768312.843188:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114878592:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768312.843199:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768312.843204:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768312/real 1775768312]  req@ffff9f1485744000 x1862022114878592/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768337 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768312.843210:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744000 x1862022114878592/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768337 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768312.843215:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768312.843216:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768312.843218:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114878592:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775768312.843220:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114878720:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768312.843229:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768312.843234:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768312/real 1775768312]  req@ffff9f1485747100 x1862022114878720/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768337 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768312.843240:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747100 x1862022114878720/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768337 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768312.843244:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768312.843245:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768312.843247:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114878720:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775768312.843251:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114878848:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768312.843265:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768312.843274:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768312/real 1775768312]  req@ffff9f1485745180 x1862022114878848/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768337 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768312.843286:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745180 x1862022114878848/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768337 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768312.843294:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768312.843297:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768312.843301:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114878848:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768312.843306:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114878976:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768312.843322:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768312.843336:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768312/real 1775768312]  req@ffff9f1485745c00 x1862022114878976/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768337 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768312.843343:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745c00 x1862022114878976/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768337 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768312.843348:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768312.843350:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768312.843351:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114878976:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768312.843368:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114879104:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768312.843382:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768312.843388:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768312/real 1775768312]  req@ffff9f1485746680 x1862022114879104/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768337 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768312.843395:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746680 x1862022114879104/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768337 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768312.843408:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768312.843411:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768312.843414:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114879104:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775768313.098739:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768313.866672:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768314.122730:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768314.890679:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775768315.146732:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768315.914648:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775768316.170716:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768316.938669:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:0.0:1775768317.194678:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1775768317.962663:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:2.0:1775768317.962668:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775768317.962674:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775768317.962679:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768317.962688:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5695
00000100:00080000:3.0:1775768317.962700:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768317.962707:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775768317.962745:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745500 x1862022114879232/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775768317.962772:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775768317.962776:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775768317.962778:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768317.962782:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5695
00000100:00080000:3.0:1775768317.962785:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768317.962790:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775768317.962800:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775768317.962810:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114879232:192.168.202.110@tcp:250:kworker.0
00000100:00100000:3.0:1775768317.962815:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744e00 x1862022114879360/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775768317.962827:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775768317.962831:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775768317.962832:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768317.962836:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5695
00000100:00080000:3.0:1775768317.962840:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768317.962843:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775768317.962849:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775768317.962858:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744380 x1862022114879488/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775768317.962867:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775768317.962869:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775768317.962870:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000800:00000100:0.0:1775768317.962875:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775768317.962876:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5695
00000100:00080000:3.0:1775768317.962880:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768317.962883:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775768317.962887:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775768317.962894:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746a00 x1862022114879616/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775768317.962900:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775768317.962902:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775768317.962903:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768317.962906:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5695
00000100:00100000:0.0:1775768317.962909:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768317/real 1775768317]  req@ffff9f1485745500 x1862022114879232/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768342 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775768317.962910:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768317.962915:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775768317.962920:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775768317.962928:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744a80 x1862022114879744/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775768317.962931:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745500 x1862022114879232/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768342 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775768317.962945:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775768317.962948:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775768317.962948:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775768317.962950:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775768317.962953:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:3.0:1775768317.962955:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5695
00000100:00080000:3.0:1775768317.962958:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1775768317.962959:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114879232:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775768317.962962:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775768317.962966:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775768317.962974:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747480 x1862022114879872/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775768317.962980:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1775768317.962981:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114879360:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775768317.962986:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775768317.962988:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768317.962991:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5695
00000100:00080000:3.0:1775768317.962993:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768317.962996:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775768317.962999:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000800:00000100:0.0:1775768317.963003:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775768317.963008:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745880 x1862022114880000/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775768317.963014:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768317/real 1775768317]  req@ffff9f1485744e00 x1862022114879360/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768342 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775768317.963016:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775768317.963022:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775768317.963025:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768317.963029:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5695
00000100:00080000:3.0:1775768317.963031:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768317.963033:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775768317.963039:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775768317.963043:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744e00 x1862022114879360/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768342 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768317.963063:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768317.963065:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775768317.963068:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746300 x1862022114880128/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775768317.963071:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114879360:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768317.963082:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114879488:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768317.963099:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768317.963106:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768317/real 1775768317]  req@ffff9f1485744380 x1862022114879488/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768342 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768317.963113:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744380 x1862022114879488/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768342 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768317.963118:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768317.963120:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768317.963121:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114879488:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768317.963124:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114879616:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768317.963133:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768317.963138:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768317/real 1775768317]  req@ffff9f1485746a00 x1862022114879616/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768342 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768317.963144:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746a00 x1862022114879616/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768342 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768317.963148:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768317.963150:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768317.963151:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114879616:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775768317.963154:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114879744:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768317.963162:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768317.963167:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768317/real 1775768317]  req@ffff9f1485744a80 x1862022114879744/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768342 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768317.963173:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744a80 x1862022114879744/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768342 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768317.963177:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768317.963178:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768317.963180:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114879744:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775768317.963189:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114879872:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768317.963197:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768317.963201:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768317/real 1775768317]  req@ffff9f1485747480 x1862022114879872/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768342 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768317.963207:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747480 x1862022114879872/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768342 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768317.963211:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768317.963213:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768317.963215:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114879872:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768317.963218:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114880000:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768317.963225:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768317.963230:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768317/real 1775768317]  req@ffff9f1485745880 x1862022114880000/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768342 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768317.963236:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745880 x1862022114880000/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768342 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768317.963241:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768317.963242:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768317.963244:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114880000:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768317.963249:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114880128:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768317.963263:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768317.963272:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768317/real 1775768317]  req@ffff9f1485746300 x1862022114880128/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768342 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768317.963282:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746300 x1862022114880128/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768342 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768317.963289:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768317.963292:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768317.963295:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114880128:192.168.202.110@tcp:8:kworker.0
00010000:00010000:1.0:1775768318.218682:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768318.986650:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775768319.242798:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768320.010652:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775768320.266686:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768321.034667:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768321.290759:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775768322.058690:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775768322.314706:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1775768323.082676:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:3.0:1775768323.082682:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:0.0:1775768323.082684:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1775768323.082690:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775768323.082698:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5700
00000100:00080000:0.0:1775768323.082707:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775768323.082712:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775768323.082750:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3100 x1862022114880256/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775768323.082773:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775768323.082776:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775768323.082777:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775768323.082780:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5700
00000100:00080000:0.0:1775768323.082783:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775768323.082785:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775768323.082794:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775768323.082803:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d1180 x1862022114880384/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775768323.082811:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114880256:192.168.202.110@tcp:250:kworker.0
00000100:00080000:0.0:1775768323.082812:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775768323.082819:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:0.0:1775768323.082821:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775768323.082825:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5700
00000100:00080000:0.0:1775768323.082826:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775768323.082828:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775768323.082831:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775768323.082839:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3b80 x1862022114880512/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775768323.082845:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775768323.082847:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775768323.082848:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775768323.082850:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5700
00000100:00080000:0.0:1775768323.082853:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775768323.082855:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775768323.082858:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775768323.082864:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d1f80 x1862022114880640/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775768323.082869:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775768323.082871:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1775768323.082872:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775768323.082874:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5700
00000100:00080000:0.0:1775768323.082876:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775768323.082879:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775768323.082882:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:2.0:1775768323.082886:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768323.082891:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0380 x1862022114880768/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775768323.082901:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775768323.082903:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775768323.082906:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775768323.082909:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5700
00000100:00080000:0.0:1775768323.082913:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775768323.082918:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775768323.082924:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775768323.082926:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768323/real 1775768323]  req@ffff9f14a04d3100 x1862022114880256/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768348 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768323.082936:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3480 x1862022114880896/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775768323.082942:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3100 x1862022114880256/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768348 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768323.082951:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775768323.082954:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:0.0:1775768323.082956:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775768323.082962:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5700
00000100:00080000:0.0:1775768323.082968:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768323.082969:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768323.082971:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775768323.082977:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00080000:2.0:1775768323.082979:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775768323.082985:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114880256:192.168.202.110@tcp:250:kworker.0
00000100:00100000:0.0:1775768323.082989:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0a80 x1862022114881024/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775768323.083008:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775768323.083011:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775768323.083012:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775768323.083016:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5700
00000100:00080000:0.0:1775768323.083018:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775768323.083021:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775768323.083024:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775768323.083030:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114880384:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768323.083035:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d2300 x1862022114881152/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:2.0:1775768323.083059:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775768323.083069:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768323/real 1775768323]  req@ffff9f14a04d1180 x1862022114880384/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768348 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775768323.083077:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1180 x1862022114880384/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768348 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775768323.083082:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775768323.083085:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775768323.083086:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114880384:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775768323.083090:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114880512:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775768323.083110:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775768323.083128:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768323/real 1775768323]  req@ffff9f14a04d3b80 x1862022114880512/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768348 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775768323.083146:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3b80 x1862022114880512/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768348 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775768323.083153:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775768323.083156:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775768323.083159:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114880512:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775768323.083167:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114880640:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775768323.083182:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775768323.083188:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768323/real 1775768323]  req@ffff9f14a04d1f80 x1862022114880640/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768348 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775768323.083195:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1f80 x1862022114880640/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768348 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775768323.083200:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775768323.083201:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775768323.083203:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114880640:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775768323.083206:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114880768:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775768323.083214:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775768323.083226:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768323/real 1775768323]  req@ffff9f14a04d0380 x1862022114880768/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768348 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775768323.083236:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0380 x1862022114880768/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768348 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775768323.083247:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775768323.083250:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775768323.083254:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114880768:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775768323.083259:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114880896:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775768323.083277:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775768323.083285:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768323/real 1775768323]  req@ffff9f14a04d3480 x1862022114880896/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768348 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775768323.083292:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3480 x1862022114880896/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768348 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775768323.083297:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775768323.083299:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775768323.083301:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114880896:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775768323.083304:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114881024:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775768323.083314:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775768323.083325:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768323/real 1775768323]  req@ffff9f14a04d0a80 x1862022114881024/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768348 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775768323.083332:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0a80 x1862022114881024/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768348 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775768323.083336:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775768323.083338:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775768323.083341:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114881024:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775768323.083361:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114881152:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775768323.083379:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775768323.083388:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768323/real 1775768323]  req@ffff9f14a04d2300 x1862022114881152/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768348 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775768323.083398:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d2300 x1862022114881152/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768348 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775768323.083404:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775768323.083406:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775768323.083408:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114881152:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775768323.338714:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775768324.106643:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:0.0:1775768324.362685:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775768325.130682:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775768325.386736:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768326.154674:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775768326.410717:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768327.178697:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775768327.434767:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1775768328.202611:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775768328.202618:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1775768328.202622:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775768328.202630:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5705
00000100:00100000:2.0:1775768328.202637:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:0.0:1775768328.202640:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775768328.202647:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775768328.202677:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d2a00 x1862022114881280/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775768328.202697:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775768328.202699:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775768328.202700:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775768328.202703:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5705
00000100:00080000:0.0:1775768328.202705:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775768328.202707:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775768328.202715:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775768328.202722:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d1500 x1862022114881408/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775768328.202729:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:2.0:1775768328.202730:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114881280:192.168.202.110@tcp:250:kworker.0
00000100:00080000:0.0:1775768328.202733:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:0.0:1775768328.202737:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775768328.202740:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5705
00000100:00080000:0.0:1775768328.202743:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775768328.202746:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775768328.202749:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775768328.202758:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0e00 x1862022114881536/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775768328.202764:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775768328.202766:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775768328.202767:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775768328.202769:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5705
00000100:00080000:0.0:1775768328.202775:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775768328.202777:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775768328.202780:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775768328.202790:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0000 x1862022114881664/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775768328.202798:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775768328.202800:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1775768328.202802:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775768328.202806:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5705
00000100:00080000:0.0:1775768328.202809:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775768328.202813:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775768328.202828:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775768328.202839:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114881408:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768328.202841:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3800 x1862022114881792/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775768328.202853:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775768328.202856:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775768328.202857:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775768328.202860:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5705
00000100:00080000:0.0:1775768328.202863:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775768328.202869:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775768328.202869:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114881536:192.168.202.110@tcp:38:kworker.0
00000008:00100000:0.0:1775768328.202873:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775768328.202883:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d2300 x1862022114881920/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775768328.202893:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775768328.202895:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:0.0:1775768328.202897:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775768328.202899:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5705
00000100:00080000:0.0:1775768328.202901:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775768328.202904:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775768328.202907:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775768328.202908:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114881664:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775768328.202916:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0a80 x1862022114882048/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775768328.202922:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775768328.202925:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775768328.202926:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775768328.202928:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5705
00000100:00080000:0.0:1775768328.202931:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:2.0:1775768328.202933:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114881792:192.168.202.110@tcp:8:kworker.0
00000100:00080000:0.0:1775768328.202938:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775768328.202944:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775768328.202964:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114881920:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768328.202970:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3480 x1862022114882176/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775768328.202981:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114882048:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775768328.202995:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114882176:192.168.202.110@tcp:8:kworker.0
00010000:00010000:3.0:1775768328.458705:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768329.226636:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775768329.482736:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768330.250674:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775768330.506728:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768331.274646:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000400:00000100:1.0:1775768331.274826:0:2390:0:(lib-socket.c:454:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.202.110:988
00000400:00000100:1.0:1775768331.274862:0:2390:0:(acceptor.c:111:lnet_connect_console_error()) Connection to 192.168.202.110@tcp at host 192.168.202.110 was unreachable: the network or that node may be down, or Lustre may be misconfigured.
00000800:00000100:1.0:1775768331.274877:0:2390:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:1.0:1775768331.274908:0:2390:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:1.0:1775768331.274927:0:2390:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:1.0:1775768331.274936:0:2390:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:1.0:1775768331.274945:0:2390:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:1.0:1775768331.274954:0:2390:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000100:00100000:2.0:1775768331.274957:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768328/real 1775768331]  req@ffff9f14a04d1500 x1862022114881408/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768353 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000800:00000100:1.0:1775768331.274964:0:2390:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:1.0:1775768331.274975:0:2390:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000100:00100000:2.0:1775768331.274991:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1500 x1862022114881408/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768353 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775768331.275003:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775768331.275008:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775768331.275011:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114881408:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775768331.275019:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768328/real 1775768331]  req@ffff9f14a04d0e00 x1862022114881536/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768353 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775768331.275031:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0e00 x1862022114881536/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768353 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775768331.275038:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775768331.275042:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775768331.275045:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114881536:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775768331.275051:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768328/real 1775768331]  req@ffff9f14a04d0000 x1862022114881664/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768353 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775768331.275063:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0000 x1862022114881664/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768353 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775768331.275070:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775768331.275073:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775768331.275076:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114881664:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775768331.275082:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768328/real 1775768331]  req@ffff9f14a04d2a00 x1862022114881280/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768353 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775768331.275094:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d2a00 x1862022114881280/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768353 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775768331.275121:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775768331.275124:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775768331.275127:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114881280:192.168.202.110@tcp:250:kworker.0
00000100:00100000:2.0:1775768331.275174:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768328/real 1775768331]  req@ffff9f14a04d3480 x1862022114882176/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768353 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775768331.275186:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3480 x1862022114882176/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768353 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775768331.275194:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775768331.275198:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775768331.275201:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114882176:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775768331.275207:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768328/real 1775768331]  req@ffff9f14a04d3800 x1862022114881792/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768353 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775768331.275219:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3800 x1862022114881792/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768353 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775768331.275226:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775768331.275229:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775768331.275232:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114881792:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775768331.275238:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768328/real 1775768331]  req@ffff9f14a04d2300 x1862022114881920/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768353 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775768331.275248:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d2300 x1862022114881920/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768353 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775768331.275254:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775768331.275257:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775768331.275259:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114881920:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775768331.275265:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768328/real 1775768331]  req@ffff9f14a04d0a80 x1862022114882048/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768353 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775768331.275275:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0a80 x1862022114882048/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768353 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775768331.275280:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775768331.275283:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775768331.275286:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114882048:192.168.202.110@tcp:38:kworker.0
00010000:00010000:0.0:1775768331.530693:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768332.298685:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775768332.554709:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1775768333.322655:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768333.322665:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00100000:3.0:1775768333.322665:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:2.0:1775768333.322670:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768333.322680:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5710
00000100:00080000:2.0:1775768333.322692:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768333.322718:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775768333.322755:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1500 x1862022114882304/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768333.322788:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768333.322792:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775768333.322794:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768333.322798:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5710
00000100:00080000:2.0:1775768333.322802:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768333.322806:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768333.322817:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775768333.322830:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0a80 x1862022114882432/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775768333.322834:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114882304:192.168.202.110@tcp:250:kworker.0
00000100:00080000:2.0:1775768333.322861:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768333.322865:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775768333.322866:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768333.322871:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5710
00000100:00080000:2.0:1775768333.322875:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768333.322879:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768333.322884:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775768333.322909:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2a00 x1862022114882560/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768333.322919:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000800:00000100:0.0:1775768333.322921:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:2.0:1775768333.322924:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775768333.322927:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768333.322932:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5710
00000100:00080000:2.0:1775768333.322936:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768333.322941:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768333.322951:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775768333.322962:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768333/real 1775768333]  req@ffff9f14852d1500 x1862022114882304/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768358 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775768333.322965:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2680 x1862022114882688/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768333.322990:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768333.322993:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1775768333.322995:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768333.323000:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5710
00000100:00100000:0.0:1775768333.323003:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1500 x1862022114882304/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768358 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775768333.323005:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768333.323013:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00080000:0.0:1775768333.323019:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000008:00100000:2.0:1775768333.323023:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00080000:0.0:1775768333.323038:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775768333.323041:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1f80 x1862022114882816/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775768333.323047:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114882304:192.168.202.110@tcp:250:kworker.0
00000100:00080000:2.0:1775768333.323056:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768333.323060:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775768333.323062:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768333.323068:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5710
00000100:00080000:2.0:1775768333.323072:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768333.323077:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775768333.323081:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114882432:192.168.202.110@tcp:38:kworker.0
00000008:00100000:2.0:1775768333.323084:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775768333.323098:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3800 x1862022114882944/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1775768333.323110:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:2.0:1775768333.323111:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768333.323116:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775768333.323117:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768333.323121:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5710
00000100:00080000:2.0:1775768333.323131:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1775768333.323136:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768333/real 1775768333]  req@ffff9f14852d0a80 x1862022114882432/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768358 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775768333.323139:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768333.323146:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775768333.323155:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0a80 x1862022114882432/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768358 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775768333.323160:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3b80 x1862022114883072/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775768333.323176:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775768333.323180:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775768333.323182:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00080000:2.0:1775768333.323185:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775768333.323189:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775768333.323190:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114882432:192.168.202.110@tcp:38:kworker.0
00000100:00080000:2.0:1775768333.323198:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5710
00000100:00100000:0.0:1775768333.323202:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114882560:192.168.202.110@tcp:38:kworker.0
00000100:00080000:2.0:1775768333.323204:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768333.323211:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768333.323217:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775768333.323227:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0380 x1862022114883200/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1775768333.323240:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768333.323252:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768333/real 1775768333]  req@ffff9f14852d2a00 x1862022114882560/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768358 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768333.323264:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2a00 x1862022114882560/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768358 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768333.323296:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768333.323299:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768333.323302:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114882560:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768333.323308:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114882688:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768333.323328:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768333.323338:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768333/real 1775768333]  req@ffff9f14852d2680 x1862022114882688/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768358 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768333.323349:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2680 x1862022114882688/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768358 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768333.323356:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768333.323359:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768333.323362:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114882688:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775768333.323367:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114882816:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768333.323385:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768333.323395:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768333/real 1775768333]  req@ffff9f14852d1f80 x1862022114882816/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768358 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768333.323406:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1f80 x1862022114882816/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768358 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768333.323413:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768333.323416:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768333.323420:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114882816:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775768333.323437:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114882944:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768333.323458:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768333.323468:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768333/real 1775768333]  req@ffff9f14852d3800 x1862022114882944/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768358 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768333.323479:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3800 x1862022114882944/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768358 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768333.323487:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768333.323490:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768333.323493:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114882944:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768333.323499:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114883072:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768333.323516:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768333.323527:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768333/real 1775768333]  req@ffff9f14852d3b80 x1862022114883072/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768358 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768333.323541:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3b80 x1862022114883072/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768358 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768333.323549:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768333.323583:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768333.323586:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114883072:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768333.323592:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114883200:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768333.323611:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768333.323622:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768333/real 1775768333]  req@ffff9f14852d0380 x1862022114883200/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768358 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768333.323634:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0380 x1862022114883200/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768358 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768333.323641:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768333.323644:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768333.323647:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114883200:192.168.202.110@tcp:8:kworker.0
00010000:00010000:1.0:1775768333.578691:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775768334.346641:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775768334.602717:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775768335.370616:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768335.626688:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768336.394687:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775768336.650691:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768337.418683:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775768337.674741:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768338.442710:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:1.0:1775768338.442741:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775768338.442752:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1775768338.442757:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775768338.442770:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5715
00000100:00080000:1.0:1775768338.442785:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775768338.442795:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775768338.442844:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fea00 x1862022114883328/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775768338.442889:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775768338.442893:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775768338.442895:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775768338.442900:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5715
00000100:00080000:1.0:1775768338.442904:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775768338.442908:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775768338.442920:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775768338.442929:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114883328:192.168.202.110@tcp:250:kworker.0
00000100:00100000:1.0:1775768338.442935:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdf80 x1862022114883456/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775768338.442946:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775768338.442950:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775768338.442952:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775768338.442956:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5715
00000100:00080000:1.0:1775768338.442960:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775768338.442964:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775768338.442969:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775768338.442981:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fe300 x1862022114883584/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775768338.442990:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775768338.442994:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775768338.442997:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775768338.443000:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5715
00000100:00080000:1.0:1775768338.443005:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000800:00000100:0.0:1775768338.443006:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:1.0:1775768338.443013:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775768338.443019:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775768338.443033:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fd880 x1862022114883712/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775768338.443043:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775768338.443047:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1775768338.443049:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775768338.443053:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5715
00000100:00080000:1.0:1775768338.443057:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1775768338.443059:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768338/real 1775768338]  req@ffff9f14ab4fea00 x1862022114883328/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768363 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775768338.443063:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775768338.443071:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775768338.443084:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fea00 x1862022114883328/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768363 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775768338.443087:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdc00 x1862022114883840/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775768338.443108:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775768338.443110:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775768338.443116:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:1.0:1775768338.443116:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775768338.443121:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775768338.443123:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114883328:192.168.202.110@tcp:250:kworker.0
00000100:00080000:1.0:1775768338.443128:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5715
00000100:00080000:1.0:1775768338.443133:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775768338.443138:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775768338.443143:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775768338.443152:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114883456:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775768338.443158:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fce00 x1862022114883968/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775768338.443169:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775768338.443173:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775768338.443175:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000800:00000100:0.0:1775768338.443182:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:1.0:1775768338.443182:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5715
00000100:00080000:1.0:1775768338.443189:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775768338.443193:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775768338.443198:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775768338.443203:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768338/real 1775768338]  req@ffff9f14ab4fdf80 x1862022114883456/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768363 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775768338.443212:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff100 x1862022114884096/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775768338.443222:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdf80 x1862022114883456/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768363 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775768338.443225:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775768338.443234:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775768338.443237:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775768338.443246:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5715
00000100:00080000:0.0:1775768338.443250:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775768338.443251:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775768338.443256:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00080000:1.0:1775768338.443259:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775768338.443263:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114883456:192.168.202.110@tcp:38:kworker.0
00000008:00100000:1.0:1775768338.443267:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775768338.443271:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114883584:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775768338.443282:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fc380 x1862022114884224/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1775768338.443293:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768338.443305:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768338/real 1775768338]  req@ffff9f14ab4fe300 x1862022114883584/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768363 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768338.443317:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fe300 x1862022114883584/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768363 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768338.443325:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768338.443328:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768338.443331:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114883584:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768338.443337:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114883712:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768338.443352:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768338.443360:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768338/real 1775768338]  req@ffff9f14ab4fd880 x1862022114883712/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768363 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768338.443371:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fd880 x1862022114883712/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768363 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768338.443378:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768338.443382:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768338.443385:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114883712:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775768338.443390:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114883840:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768338.443404:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768338.443412:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768338/real 1775768338]  req@ffff9f14ab4fdc00 x1862022114883840/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768363 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768338.443422:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdc00 x1862022114883840/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768363 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768338.443430:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768338.443432:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768338.443435:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114883840:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775768338.443449:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114883968:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768338.443464:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768338.443473:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768338/real 1775768338]  req@ffff9f14ab4fce00 x1862022114883968/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768363 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768338.443484:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fce00 x1862022114883968/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768363 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768338.443491:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768338.443494:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768338.443497:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114883968:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768338.443502:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114884096:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768338.443515:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768338.443522:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768338/real 1775768338]  req@ffff9f14ab4ff100 x1862022114884096/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768363 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768338.443533:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff100 x1862022114884096/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768363 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768338.443540:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768338.443543:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768338.443546:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114884096:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768338.443551:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114884224:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768338.443614:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768338.443623:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768338/real 1775768338]  req@ffff9f14ab4fc380 x1862022114884224/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768363 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768338.443634:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fc380 x1862022114884224/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768363 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768338.443642:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768338.443645:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768338.443648:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114884224:192.168.202.110@tcp:8:kworker.0
00010000:00010000:0.0:1775768338.698756:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768339.466664:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768339.722762:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768340.490689:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775768340.746746:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768341.514678:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775768341.770761:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768342.538651:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768342.794749:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768343.562685:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:2.0:1775768343.562732:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768343.562738:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1775768343.562742:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768343.562751:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5721
00000100:00080000:2.0:1775768343.562762:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768343.562770:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775768343.562820:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0700 x1862022114884352/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768343.562862:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768343.562866:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775768343.562868:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768343.562873:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5721
00000100:00080000:2.0:1775768343.562878:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768343.562882:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768343.562892:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775768343.562906:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2d80 x1862022114884480/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768343.562917:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768343.562920:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775768343.562922:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768343.562926:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5721
00000100:00080000:2.0:1775768343.562930:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768343.562933:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768343.562937:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775768343.562946:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0000 x1862022114884608/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768343.562955:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768343.562958:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775768343.562959:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768343.562962:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5721
00000100:00080000:2.0:1775768343.562965:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768343.562968:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768343.562973:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775768343.562981:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1180 x1862022114884736/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768343.562989:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768343.562991:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1775768343.562993:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768343.562996:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5721
00000100:00080000:2.0:1775768343.562999:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768343.563002:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768343.563005:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775768343.563014:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1880 x1862022114884864/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768343.563022:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768343.563025:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775768343.563026:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768343.563029:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5721
00000100:00080000:2.0:1775768343.563032:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768343.563063:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768343.563067:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775768343.563077:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0e00 x1862022114884992/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768343.563085:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768343.563088:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775768343.563090:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768343.563093:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5721
00000100:00080000:2.0:1775768343.563097:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768343.563100:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768343.563104:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775768343.563112:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2300 x1862022114885120/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768343.563121:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768343.563124:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775768343.563125:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768343.563128:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5721
00000100:00080000:2.0:1775768343.563131:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768343.563134:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768343.563138:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775768343.563147:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3100 x1862022114885248/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775768343.563322:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114884352:192.168.202.110@tcp:250:kworker.0
00000800:00000100:3.0:1775768343.563442:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775768343.563642:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768343/real 1775768343]  req@ffff9f14852d0700 x1862022114884352/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768368 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775768343.563660:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0700 x1862022114884352/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768368 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775768343.563670:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775768343.563674:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775768343.563678:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114884352:192.168.202.110@tcp:250:kworker.0
00000100:00100000:3.0:1775768343.563685:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114884480:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775768343.563708:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775768343.563720:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768343/real 1775768343]  req@ffff9f14852d2d80 x1862022114884480/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768368 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775768343.563730:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2d80 x1862022114884480/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768368 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775768343.563743:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775768343.563747:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775768343.563750:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114884480:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775768343.563756:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114884608:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775768343.563776:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775768343.563785:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768343/real 1775768343]  req@ffff9f14852d0000 x1862022114884608/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768368 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775768343.563796:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0000 x1862022114884608/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768368 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775768343.563817:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775768343.563820:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775768343.563822:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114884608:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775768343.563829:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114884736:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775768343.563848:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775768343.563858:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768343/real 1775768343]  req@ffff9f14852d1180 x1862022114884736/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768368 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775768343.563870:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1180 x1862022114884736/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768368 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775768343.563876:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775768343.563879:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775768343.563882:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114884736:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775768343.563887:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114884864:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775768343.563906:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775768343.563915:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768343/real 1775768343]  req@ffff9f14852d1880 x1862022114884864/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768368 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775768343.563925:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1880 x1862022114884864/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768368 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775768343.563932:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775768343.563934:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775768343.563937:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114884864:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775768343.563942:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114884992:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775768343.563959:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775768343.563968:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768343/real 1775768343]  req@ffff9f14852d0e00 x1862022114884992/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768368 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775768343.563978:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0e00 x1862022114884992/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768368 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775768343.563984:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775768343.563987:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775768343.563990:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114884992:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775768343.563995:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114885120:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775768343.564012:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775768343.564020:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768343/real 1775768343]  req@ffff9f14852d2300 x1862022114885120/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768368 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775768343.564030:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2300 x1862022114885120/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768368 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775768343.564037:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775768343.564041:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775768343.564043:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114885120:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775768343.564048:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114885248:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775768343.564064:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775768343.564073:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768343/real 1775768343]  req@ffff9f14852d3100 x1862022114885248/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768368 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775768343.564083:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3100 x1862022114885248/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768368 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775768343.564090:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775768343.564093:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775768343.564096:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114885248:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775768343.818773:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768344.586657:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775768344.844802:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768345.610881:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:0.0:1775768345.867357:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768346.634683:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768346.890713:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768347.658873:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775768347.914724:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768348.682798:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:1.0:1775768348.682863:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775768348.682868:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1775768348.682873:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775768348.682883:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5726
00000100:00080000:1.0:1775768348.682892:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775768348.682898:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775768348.683126:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fca80 x1862022114885376/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775768348.683146:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775768348.683150:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775768348.683152:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775768348.683156:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5726
00000100:00080000:1.0:1775768348.683161:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775768348.683165:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775768348.683174:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775768348.683186:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ffb80 x1862022114885504/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775768348.683197:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:3.0:1775768348.683201:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114885376:192.168.202.110@tcp:250:kworker.0
00000100:00080000:1.0:1775768348.683202:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775768348.683205:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775768348.683212:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5726
00000100:00080000:1.0:1775768348.683216:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775768348.683220:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775768348.683224:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775768348.683236:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fed80 x1862022114885632/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775768348.683251:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775768348.683255:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775768348.683256:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775768348.683259:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5726
00000100:00080000:1.0:1775768348.683262:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775768348.683266:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000800:00000100:3.0:1775768348.683268:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000008:00100000:1.0:1775768348.683271:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775768348.683280:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff800 x1862022114885760/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775768348.683287:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775768348.683289:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1775768348.683290:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775768348.683292:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5726
00000100:00080000:1.0:1775768348.683294:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775768348.683296:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775768348.683298:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775768348.683308:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fc380 x1862022114885888/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775768348.683310:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768348/real 1775768348]  req@ffff9f14ab4fca80 x1862022114885376/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768373 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775768348.683321:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775768348.683328:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775768348.683330:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775768348.683336:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5726
00000100:00080000:1.0:1775768348.683338:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775768348.683341:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775768348.683344:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775768348.683348:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fca80 x1862022114885376/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768373 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775768348.683351:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff100 x1862022114886016/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775768348.683380:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775768348.683382:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775768348.683383:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768348.683383:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775768348.683387:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5726
00000100:00080000:1.0:1775768348.683390:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775768348.683394:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00080000:3.0:1775768348.683394:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000008:00100000:1.0:1775768348.683399:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775768348.683403:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114885376:192.168.202.110@tcp:250:kworker.0
00000100:00100000:1.0:1775768348.683408:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fce00 x1862022114886144/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775768348.683417:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775768348.683420:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775768348.683421:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775768348.683425:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5726
00000100:00080000:1.0:1775768348.683429:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:3.0:1775768348.683430:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ffb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114885504:192.168.202.110@tcp:38:kworker.0
00000100:00080000:1.0:1775768348.683435:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775768348.683441:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775768348.683453:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdc00 x1862022114886272/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:3.0:1775768348.683462:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775768348.683477:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768348/real 1775768348]  req@ffff9f14ab4ffb80 x1862022114885504/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768373 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775768348.683491:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ffb80 x1862022114885504/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768373 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775768348.683498:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775768348.683501:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775768348.683503:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ffb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114885504:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775768348.683509:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114885632:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775768348.683524:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775768348.683532:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768348/real 1775768348]  req@ffff9f14ab4fed80 x1862022114885632/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768373 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775768348.683543:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fed80 x1862022114885632/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768373 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775768348.683549:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775768348.683552:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775768348.683579:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114885632:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775768348.683585:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114885760:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775768348.683603:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775768348.683613:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768348/real 1775768348]  req@ffff9f14ab4ff800 x1862022114885760/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768373 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775768348.683623:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff800 x1862022114885760/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768373 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775768348.683630:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775768348.683633:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775768348.683636:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114885760:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775768348.683641:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114885888:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775768348.683659:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775768348.683667:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768348/real 1775768348]  req@ffff9f14ab4fc380 x1862022114885888/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768373 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775768348.683677:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fc380 x1862022114885888/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768373 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775768348.683684:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775768348.683686:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775768348.683689:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114885888:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775768348.683694:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114886016:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775768348.683707:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775768348.683724:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768348/real 1775768348]  req@ffff9f14ab4ff100 x1862022114886016/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768373 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775768348.683735:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff100 x1862022114886016/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768373 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775768348.683742:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775768348.683745:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775768348.683762:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114886016:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775768348.683768:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114886144:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775768348.683786:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775768348.683795:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768348/real 1775768348]  req@ffff9f14ab4fce00 x1862022114886144/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768373 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775768348.683806:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fce00 x1862022114886144/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768373 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775768348.683814:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775768348.683816:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775768348.683819:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114886144:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775768348.683837:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114886272:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775768348.683855:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775768348.683865:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768348/real 1775768348]  req@ffff9f14ab4fdc00 x1862022114886272/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768373 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775768348.683876:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdc00 x1862022114886272/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768373 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775768348.683889:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775768348.683892:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775768348.683895:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114886272:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775768348.940275:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768349.706677:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768349.962732:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768350.730680:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775768350.986707:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768351.754667:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775768352.011122:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768352.778656:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:0.0:1775768353.034717:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1775768353.802661:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:2.0:1775768353.802662:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775768353.802670:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775768353.802674:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768353.802681:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5731
00000100:00080000:3.0:1775768353.802692:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768353.802837:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775768353.802871:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747b80 x1862022114886400/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775768353.802891:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775768353.802895:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775768353.802896:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768353.802900:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5731
00000100:00080000:3.0:1775768353.802913:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768353.802917:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775768353.802926:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775768353.802932:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114886400:192.168.202.110@tcp:250:kworker.0
00000100:00100000:3.0:1775768353.802943:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746300 x1862022114886528/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775768353.802951:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775768353.802954:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775768353.802956:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768353.802959:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5731
00000100:00080000:3.0:1775768353.802962:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768353.802965:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775768353.802969:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775768353.802976:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745880 x1862022114886656/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775768353.802983:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775768353.802986:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000800:00000100:0.0:1775768353.802987:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775768353.802988:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768353.802991:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5731
00000100:00080000:3.0:1775768353.802993:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768353.802996:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775768353.803000:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775768353.803008:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747480 x1862022114886784/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775768353.803016:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775768353.803019:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775768353.803021:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768353.803024:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5731
00000100:00080000:3.0:1775768353.803027:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768353.803030:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775768353.803031:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768353/real 1775768353]  req@ffff9f1485747b80 x1862022114886400/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768378 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:3.0:1775768353.803035:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775768353.803042:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744a80 x1862022114886912/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775768353.803050:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1775768353.803053:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747b80 x1862022114886400/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768378 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775768353.803053:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775768353.803055:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768353.803058:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5731
00000100:00080000:3.0:1775768353.803060:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775768353.803062:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775768353.803063:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00080000:0.0:1775768353.803066:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000008:00100000:3.0:1775768353.803067:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775768353.803070:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114886400:192.168.202.110@tcp:250:kworker.0
00000100:00100000:3.0:1775768353.803074:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746a00 x1862022114887040/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775768353.803081:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775768353.803083:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775768353.803084:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768353.803087:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5731
00000100:00100000:0.0:1775768353.803089:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114886528:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775768353.803090:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768353.803093:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775768353.803097:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775768353.803104:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744380 x1862022114887168/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775768353.803112:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775768353.803115:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775768353.803117:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768353.803120:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5731
00000100:00080000:3.0:1775768353.803123:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768353.803126:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775768353.803131:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775768353.803149:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744e00 x1862022114887296/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1775768353.803278:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768353.803290:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768353/real 1775768353]  req@ffff9f1485746300 x1862022114886528/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768378 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768353.803298:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746300 x1862022114886528/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768378 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768353.803303:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768353.803305:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768353.803307:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114886528:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768353.803311:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114886656:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768353.803325:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768353.803330:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768353/real 1775768353]  req@ffff9f1485745880 x1862022114886656/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768378 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768353.803337:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745880 x1862022114886656/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768378 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768353.803344:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768353.803346:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768353.803348:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114886656:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768353.803351:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114886784:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768353.803366:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768353.803373:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768353/real 1775768353]  req@ffff9f1485747480 x1862022114886784/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768378 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768353.803396:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747480 x1862022114886784/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768378 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768353.803404:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768353.803407:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768353.803411:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114886784:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775768353.803416:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114886912:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768353.803435:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768353.803445:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768353/real 1775768353]  req@ffff9f1485744a80 x1862022114886912/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768378 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768353.803457:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744a80 x1862022114886912/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768378 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768353.803464:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768353.803467:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768353.803471:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114886912:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775768353.803476:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114887040:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768353.803494:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768353.803503:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768353/real 1775768353]  req@ffff9f1485746a00 x1862022114887040/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768378 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768353.803514:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746a00 x1862022114887040/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768378 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768353.803522:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768353.803524:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768353.803527:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114887040:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768353.803545:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114887168:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768353.803600:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768353.803610:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768353/real 1775768353]  req@ffff9f1485744380 x1862022114887168/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768378 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768353.803621:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744380 x1862022114887168/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768378 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768353.803629:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768353.803631:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768353.803634:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114887168:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768353.803640:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114887296:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768353.803655:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768353.803665:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768353/real 1775768353]  req@ffff9f1485744e00 x1862022114887296/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768378 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768353.803676:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744e00 x1862022114887296/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768378 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768353.803707:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768353.803709:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768353.803712:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114887296:192.168.202.110@tcp:8:kworker.0
00010000:00010000:3.0:1775768354.058741:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768354.826680:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775768355.082856:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768355.850748:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775768356.106956:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768356.875202:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768357.130680:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768357.898727:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:0.0:1775768358.154812:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1775768358.922729:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775768358.922743:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1775768358.922749:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775768358.922760:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5736
00000100:00080000:1.0:1775768358.922774:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775768358.922782:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775768358.922824:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fd880 x1862022114887424/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775768358.922845:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:1.0:1775768358.922849:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775768358.922858:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775768358.922860:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775768358.922874:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5736
00000100:00080000:1.0:1775768358.922879:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775768358.922884:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775768358.922898:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775768358.922902:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114887424:192.168.202.110@tcp:250:kworker.0
00000100:00100000:1.0:1775768358.922916:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fe300 x1862022114887552/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775768358.922928:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775768358.922931:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775768358.922933:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775768358.922937:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5736
00000100:00080000:1.0:1775768358.922941:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775768358.922945:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775768358.922951:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775768358.922961:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdf80 x1862022114887680/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775768358.922969:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775768358.922973:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775768358.922974:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775768358.922978:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5736
00000100:00080000:1.0:1775768358.922981:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775768358.922987:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000800:00000100:0.0:1775768358.922988:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000008:00100000:1.0:1775768358.922993:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775768358.923004:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fea00 x1862022114887808/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775768358.923012:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775768358.923015:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1775768358.923017:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775768358.923021:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5736
00000100:00080000:1.0:1775768358.923025:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775768358.923029:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775768358.923034:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775768358.923045:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdc00 x1862022114887936/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775768358.923053:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768358/real 1775768358]  req@ffff9f14ab4fd880 x1862022114887424/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768383 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775768358.923058:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775768358.923066:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775768358.923069:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775768358.923075:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5736
00000100:00080000:1.0:1775768358.923080:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1775768358.923085:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fd880 x1862022114887424/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768383 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775768358.923086:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775768358.923095:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775768358.923108:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fce00 x1862022114888064/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775768358.923111:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768358.923121:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:1.0:1775768358.923124:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1775768358.923128:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114887424:192.168.202.110@tcp:250:kworker.0
00000100:00080000:1.0:1775768358.923129:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775768358.923133:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775768358.923139:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5736
00000100:00080000:1.0:1775768358.923144:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775768358.923147:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775768358.923153:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775768358.923161:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114887552:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775768358.923166:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff100 x1862022114888192/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775768358.923174:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775768358.923177:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775768358.923178:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775768358.923181:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5736
00000100:00080000:1.0:1775768358.923188:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775768358.923193:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000800:00000100:0.0:1775768358.923194:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000008:00100000:1.0:1775768358.923200:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775768358.923209:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fc380 x1862022114888320/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775768358.923214:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768358/real 1775768358]  req@ffff9f14ab4fe300 x1862022114887552/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768383 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768358.923228:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fe300 x1862022114887552/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768383 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768358.923237:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768358.923241:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768358.923244:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114887552:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768358.923250:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114887680:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768358.923269:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768358.923279:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768358/real 1775768358]  req@ffff9f14ab4fdf80 x1862022114887680/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768383 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768358.923288:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdf80 x1862022114887680/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768383 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768358.923294:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768358.923297:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768358.923299:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114887680:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768358.923305:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114887808:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768358.923322:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768358.923330:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768358/real 1775768358]  req@ffff9f14ab4fea00 x1862022114887808/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768383 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768358.923351:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fea00 x1862022114887808/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768383 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768358.923361:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768358.923363:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768358.923367:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114887808:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775768358.923372:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114887936:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768358.923391:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768358.923401:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768358/real 1775768358]  req@ffff9f14ab4fdc00 x1862022114887936/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768383 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768358.923410:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdc00 x1862022114887936/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768383 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768358.923418:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768358.923421:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768358.923424:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114887936:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775768358.923430:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114888064:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768358.923450:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768358.923461:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768358/real 1775768358]  req@ffff9f14ab4fce00 x1862022114888064/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768383 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768358.923473:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fce00 x1862022114888064/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768383 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768358.923481:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768358.923484:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768358.923487:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114888064:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768358.923505:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114888192:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768358.923522:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768358.923529:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768358/real 1775768358]  req@ffff9f14ab4ff100 x1862022114888192/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768383 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768358.923548:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff100 x1862022114888192/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768383 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768358.923576:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768358.923579:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768358.923583:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114888192:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768358.923588:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114888320:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768358.923606:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768358.923614:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768358/real 1775768358]  req@ffff9f14ab4fc380 x1862022114888320/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768383 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768358.923621:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fc380 x1862022114888320/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768383 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768358.923626:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768358.923628:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768358.923629:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114888320:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775768359.178747:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768359.946705:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775768360.202719:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768360.970689:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768361.226748:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768361.994700:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775768362.250789:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768363.018735:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775768363.274892:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775768364.044742:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:2.0:1775768364.044805:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768364.044815:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1775768364.044819:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768364.044831:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5741
00000100:00080000:2.0:1775768364.044844:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768364.044853:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775768364.044891:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3480 x1862022114888448/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768364.044922:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768364.044926:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775768364.044927:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768364.044932:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5741
00000100:00080000:2.0:1775768364.044936:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768364.044940:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768364.044951:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775768364.044965:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1c00 x1862022114888576/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775768364.044979:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114888448:192.168.202.110@tcp:250:kworker.0
00000100:00080000:2.0:1775768364.044989:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768364.044993:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775768364.044995:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768364.045001:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5741
00000100:00080000:2.0:1775768364.045006:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768364.045010:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768364.045016:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775768364.045030:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3100 x1862022114888704/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768364.045040:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768364.045043:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775768364.045045:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768364.045049:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5741
00000800:00000100:0.0:1775768364.045056:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:2.0:1775768364.045057:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768364.045064:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768364.045070:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775768364.045083:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2300 x1862022114888832/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768364.045093:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768364.045097:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1775768364.045099:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768364.045104:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5741
00000100:00080000:2.0:1775768364.045108:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1775768364.045110:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768364/real 1775768364]  req@ffff9f14852d3480 x1862022114888448/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768389 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775768364.045114:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768364.045119:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775768364.045129:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0e00 x1862022114888960/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768364.045137:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768364.045140:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775768364.045142:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768364.045146:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5741
00000100:00080000:2.0:1775768364.045148:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768364.045152:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768364.045155:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775768364.045163:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1880 x1862022114889088/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768364.045182:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768364.045185:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775768364.045187:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768364.045192:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5741
00000100:00080000:2.0:1775768364.045196:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768364.045200:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768364.045204:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775768364.045212:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1180 x1862022114889216/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768364.045226:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768364.045229:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775768364.045231:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768364.045235:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5741
00000100:00080000:2.0:1775768364.045238:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768364.045241:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768364.045246:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775768364.045254:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0000 x1862022114889344/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775768364.045317:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3480 x1862022114888448/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768389 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768364.045329:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768364.045333:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768364.045337:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114888448:192.168.202.110@tcp:250:kworker.0
00000100:00100000:0.0:1775768364.045357:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114888576:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768364.045381:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768364.045393:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768364/real 1775768364]  req@ffff9f14852d1c00 x1862022114888576/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768389 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768364.045404:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1c00 x1862022114888576/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768389 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768364.045412:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768364.045416:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768364.045419:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114888576:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768364.045425:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114888704:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768364.045441:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768364.045449:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768364/real 1775768364]  req@ffff9f14852d3100 x1862022114888704/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768389 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768364.045459:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3100 x1862022114888704/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768389 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768364.045467:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768364.045470:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768364.045473:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114888704:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768364.045479:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114888832:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768364.045493:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768364.045501:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768364/real 1775768364]  req@ffff9f14852d2300 x1862022114888832/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768389 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768364.045512:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2300 x1862022114888832/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768389 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768364.045519:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768364.045522:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768364.045525:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114888832:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775768364.045531:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114888960:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768364.045544:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768364.045552:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768364/real 1775768364]  req@ffff9f14852d0e00 x1862022114888960/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768389 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768364.045592:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0e00 x1862022114888960/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768389 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768364.045610:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768364.045614:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768364.045617:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114888960:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775768364.045624:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114889088:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768364.045641:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768364.045650:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768364/real 1775768364]  req@ffff9f14852d1880 x1862022114889088/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768389 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768364.045661:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1880 x1862022114889088/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768389 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768364.045668:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768364.045671:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768364.045674:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114889088:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768364.045680:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114889216:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768364.045693:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768364.045700:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768364/real 1775768364]  req@ffff9f14852d1180 x1862022114889216/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768389 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768364.045711:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1180 x1862022114889216/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768389 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768364.045718:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768364.045721:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768364.045724:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114889216:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768364.045729:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114889344:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768364.045743:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768364.045751:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768364/real 1775768364]  req@ffff9f14852d0000 x1862022114889344/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768389 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768364.045760:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0000 x1862022114889344/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768389 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768364.045773:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768364.045777:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768364.045779:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114889344:192.168.202.110@tcp:8:kworker.0
00010000:00010000:3.0:1775768364.298752:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768365.066749:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775768365.322681:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768366.090663:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775768366.346732:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768367.114671:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775768367.370729:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768368.138664:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768368.394739:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768369.162673:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:0.0:1775768369.162677:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775768369.162683:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1775768369.162686:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775768369.162694:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5746
00000100:00080000:0.0:1775768369.162704:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775768369.162710:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775768369.162743:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0380 x1862022114889472/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775768369.162766:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775768369.162769:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775768369.162770:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775768369.162772:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5746
00000100:00080000:0.0:1775768369.162774:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775768369.162777:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775768369.162785:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775768369.162798:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d1f80 x1862022114889600/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775768369.162808:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775768369.162812:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:0.0:1775768369.162815:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:3.0:1775768369.162816:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114889472:192.168.202.110@tcp:250:kworker.0
00000100:00080000:0.0:1775768369.162821:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5746
00000100:00080000:0.0:1775768369.162826:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775768369.162830:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775768369.162834:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775768369.162844:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3b80 x1862022114889728/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775768369.162850:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775768369.162853:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775768369.162854:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775768369.162856:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5746
00000100:00080000:0.0:1775768369.162859:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775768369.162862:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775768369.162865:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775768369.162871:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d1180 x1862022114889856/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:3.0:1775768369.162875:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:0.0:1775768369.162884:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775768369.162888:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1775768369.162890:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775768369.162895:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5746
00000100:00080000:0.0:1775768369.162899:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775768369.162903:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775768369.162908:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775768369.162908:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768369/real 1775768369]  req@ffff9f14a04d0380 x1862022114889472/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768394 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768369.162925:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3100 x1862022114889984/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775768369.162927:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0380 x1862022114889472/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768394 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768369.162949:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775768369.162950:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768369.162955:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775768369.162956:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768369.162964:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:0.0:1775768369.162967:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5746
00000100:00100000:3.0:1775768369.162968:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114889472:192.168.202.110@tcp:250:kworker.0
00000100:00080000:0.0:1775768369.162972:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775768369.162977:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775768369.162981:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775768369.162989:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114889600:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768369.162990:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0380 x1862022114890112/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775768369.163000:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775768369.163002:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:0.0:1775768369.163003:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775768369.163005:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5746
00000100:00080000:0.0:1775768369.163007:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775768369.163009:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775768369.163013:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000800:00000100:3.0:1775768369.163022:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768369.163024:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0a80 x1862022114890240/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775768369.163034:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:3.0:1775768369.163039:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768369/real 1775768369]  req@ffff9f14a04d1f80 x1862022114889600/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768394 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768369.163040:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775768369.163043:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775768369.163051:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5746
00000100:00080000:0.0:1775768369.163058:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775768369.163062:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775768369.163066:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775768369.163068:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1f80 x1862022114889600/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768394 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768369.163077:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d2300 x1862022114890368/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775768369.163080:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775768369.163088:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775768369.163093:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114889600:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775768369.163101:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114889728:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775768369.163122:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775768369.163132:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768369/real 1775768369]  req@ffff9f14a04d3b80 x1862022114889728/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768394 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775768369.163139:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3b80 x1862022114889728/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768394 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775768369.163144:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775768369.163146:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775768369.163148:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114889728:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775768369.163151:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114889856:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775768369.163163:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775768369.163175:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768369/real 1775768369]  req@ffff9f14a04d1180 x1862022114889856/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768394 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775768369.163182:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1180 x1862022114889856/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768394 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775768369.163187:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775768369.163189:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775768369.163191:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114889856:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775768369.163194:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114889984:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775768369.163203:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775768369.163208:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768369/real 1775768369]  req@ffff9f14a04d3100 x1862022114889984/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768394 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775768369.163214:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3100 x1862022114889984/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768394 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775768369.163219:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775768369.163222:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775768369.163225:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114889984:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775768369.163241:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114890112:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775768369.163259:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775768369.163277:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768369/real 1775768369]  req@ffff9f14a04d0380 x1862022114890112/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768394 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775768369.163284:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0380 x1862022114890112/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768394 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775768369.163289:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775768369.163291:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775768369.163293:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114890112:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775768369.163296:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114890240:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775768369.163308:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775768369.163314:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768369/real 1775768369]  req@ffff9f14a04d0a80 x1862022114890240/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768394 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775768369.163320:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0a80 x1862022114890240/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768394 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775768369.163324:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775768369.163325:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775768369.163327:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114890240:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775768369.163330:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114890368:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775768369.163345:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775768369.163351:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768369/real 1775768369]  req@ffff9f14a04d2300 x1862022114890368/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768394 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775768369.163357:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d2300 x1862022114890368/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768394 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775768369.163361:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775768369.163363:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775768369.163365:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114890368:192.168.202.110@tcp:8:kworker.0
00010000:00010000:1.0:1775768369.418712:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768370.186656:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775768370.442741:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768371.210674:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768371.466725:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768372.234645:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775768372.490707:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768373.258712:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775768373.514734:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768374.282694:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:1.0:1775768374.282711:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775768374.282718:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1775768374.282723:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775768374.282734:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5751
00000100:00080000:1.0:1775768374.282749:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775768374.282757:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775768374.282802:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff800 x1862022114890496/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775768374.282833:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775768374.282836:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775768374.282838:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775768374.282841:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5751
00000100:00080000:1.0:1775768374.282844:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775768374.282847:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775768374.282855:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775768374.282866:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fed80 x1862022114890624/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775768374.282874:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775768374.282877:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775768374.282878:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775768374.282881:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5751
00000100:00100000:3.0:1775768374.282886:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114890496:192.168.202.110@tcp:250:kworker.0
00000100:00080000:1.0:1775768374.282889:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775768374.282895:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775768374.282898:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775768374.282906:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ffb80 x1862022114890752/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775768374.282912:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775768374.282915:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775768374.282916:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775768374.282918:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5751
00000100:00080000:1.0:1775768374.282921:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775768374.282923:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775768374.282926:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775768374.282934:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fca80 x1862022114890880/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775768374.282939:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775768374.282941:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1775768374.282943:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775768374.282945:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5751
00000100:00080000:1.0:1775768374.282953:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000800:00000100:3.0:1775768374.282958:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:1.0:1775768374.282961:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775768374.282967:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775768374.282980:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fc380 x1862022114891008/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775768374.282990:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775768374.282994:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775768374.282996:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775768374.283001:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5751
00000100:00080000:1.0:1775768374.283005:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:3.0:1775768374.283008:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768374/real 1775768374]  req@ffff9f14ab4ff800 x1862022114890496/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768399 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775768374.283018:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775768374.283024:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775768374.283032:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff800 x1862022114890496/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768399 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775768374.283039:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff100 x1862022114891136/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775768374.283043:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775768374.283054:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775768374.283059:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:1.0:1775768374.283060:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775768374.283064:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:3.0:1775768374.283066:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114890496:192.168.202.110@tcp:250:kworker.0
00000100:00080000:1.0:1775768374.283071:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5751
00000100:00100000:3.0:1775768374.283075:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114890624:192.168.202.110@tcp:38:kworker.0
00000100:00080000:1.0:1775768374.283077:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775768374.283082:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775768374.283087:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775768374.283098:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fce00 x1862022114891264/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:3.0:1775768374.283102:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:1.0:1775768374.283114:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:3.0:1775768374.283115:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768374/real 1775768374]  req@ffff9f14ab4fed80 x1862022114890624/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768399 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775768374.283122:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775768374.283125:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775768374.283132:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5751
00000100:00080000:1.0:1775768374.283135:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:3.0:1775768374.283136:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fed80 x1862022114890624/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768399 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775768374.283139:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775768374.283145:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00080000:3.0:1775768374.283148:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00100000:1.0:1775768374.283155:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdc00 x1862022114891392/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775768374.283155:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775768374.283163:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114890624:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775768374.283192:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ffb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114890752:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775768374.283211:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775768374.283219:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768374/real 1775768374]  req@ffff9f14ab4ffb80 x1862022114890752/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768399 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775768374.283231:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ffb80 x1862022114890752/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768399 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775768374.283236:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775768374.283238:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775768374.283240:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ffb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114890752:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775768374.283244:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114890880:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775768374.283255:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775768374.283260:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768374/real 1775768374]  req@ffff9f14ab4fca80 x1862022114890880/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768399 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775768374.283267:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fca80 x1862022114890880/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768399 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775768374.283272:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775768374.283273:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775768374.283275:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114890880:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775768374.283278:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114891008:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775768374.283287:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775768374.283292:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768374/real 1775768374]  req@ffff9f14ab4fc380 x1862022114891008/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768399 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775768374.283298:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fc380 x1862022114891008/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768399 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775768374.283307:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775768374.283309:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775768374.283310:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114891008:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775768374.283313:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114891136:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775768374.283322:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775768374.283326:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768374/real 1775768374]  req@ffff9f14ab4ff100 x1862022114891136/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768399 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775768374.283332:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff100 x1862022114891136/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768399 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775768374.283336:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775768374.283338:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775768374.283340:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114891136:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775768374.283343:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114891264:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775768374.283351:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775768374.283356:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768374/real 1775768374]  req@ffff9f14ab4fce00 x1862022114891264/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768399 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775768374.283361:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fce00 x1862022114891264/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768399 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775768374.283366:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775768374.283367:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775768374.283370:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114891264:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775768374.283373:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114891392:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775768374.283383:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775768374.283387:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768374/real 1775768374]  req@ffff9f14ab4fdc00 x1862022114891392/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768399 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775768374.283393:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdc00 x1862022114891392/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768399 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775768374.283398:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775768374.283399:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775768374.283401:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114891392:192.168.202.110@tcp:8:kworker.0
00010000:00010000:1.0:1775768374.538768:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768375.306672:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768375.562729:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768376.330641:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775768376.586727:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768377.354661:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775768377.610720:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768378.378629:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768378.635713:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768379.402691:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:1.0:1775768379.403617:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775768379.403624:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1775768379.403629:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775768379.403640:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5756
00000100:00080000:1.0:1775768379.403653:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775768379.403660:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775768379.403699:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fea00 x1862022114891520/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775768379.403724:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775768379.403728:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775768379.403730:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775768379.403734:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5756
00000100:00080000:1.0:1775768379.403739:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775768379.403746:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775768379.403747:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114891520:192.168.202.110@tcp:250:kworker.0
00000008:00100000:1.0:1775768379.403758:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775768379.403774:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdf80 x1862022114891648/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775768379.403785:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775768379.403789:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775768379.403791:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775768379.403795:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5756
00000100:00080000:1.0:1775768379.403799:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775768379.403802:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775768379.403808:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775768379.403819:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fe300 x1862022114891776/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:3.0:1775768379.403828:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:1.0:1775768379.403835:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775768379.403838:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775768379.403840:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775768379.403846:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5756
00000100:00080000:1.0:1775768379.403850:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775768379.403853:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775768379.403858:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775768379.403872:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768379/real 1775768379]  req@ffff9f14ab4fea00 x1862022114891520/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768404 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775768379.403873:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fd880 x1862022114891904/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775768379.403907:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775768379.403911:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00100000:3.0:1775768379.403911:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fea00 x1862022114891520/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768404 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775768379.403914:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775768379.403922:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5756
00000100:00080000:1.0:1775768379.403928:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775768379.403933:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00080000:3.0:1775768379.403933:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000008:00100000:1.0:1775768379.403941:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00080000:3.0:1775768379.403942:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775768379.403949:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114891520:192.168.202.110@tcp:250:kworker.0
00000100:00100000:1.0:1775768379.403956:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdc00 x1862022114892032/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775768379.403965:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775768379.403969:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775768379.403971:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:3.0:1775768379.403976:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114891648:192.168.202.110@tcp:38:kworker.0
00000100:00080000:1.0:1775768379.403977:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5756
00000100:00080000:1.0:1775768379.403982:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775768379.403987:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775768379.403993:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775768379.404004:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fce00 x1862022114892160/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775768379.404014:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775768379.404018:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775768379.404019:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775768379.404023:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5756
00000800:00000100:3.0:1775768379.404104:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:1.0:1775768379.404178:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775768379.404184:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775768379.404189:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775768379.404200:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff100 x1862022114892288/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775768379.404210:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775768379.404213:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775768379.404215:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775768379.404219:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5756
00000100:00080000:1.0:1775768379.404223:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775768379.404226:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775768379.404231:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775768379.404241:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fc380 x1862022114892416/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775768379.406215:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768379/real 1775768379]  req@ffff9f14ab4fdf80 x1862022114891648/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768404 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775768379.406235:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdf80 x1862022114891648/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768404 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775768379.406245:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775768379.406249:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775768379.406252:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114891648:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775768379.406259:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114891776:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775768379.406292:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775768379.406304:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768379/real 1775768379]  req@ffff9f14ab4fe300 x1862022114891776/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768404 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775768379.406315:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fe300 x1862022114891776/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768404 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775768379.406323:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775768379.406326:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775768379.406330:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114891776:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775768379.406336:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114891904:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775768379.406356:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775768379.406364:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768379/real 1775768379]  req@ffff9f14ab4fd880 x1862022114891904/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768404 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775768379.406376:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fd880 x1862022114891904/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768404 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775768379.406381:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775768379.406383:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775768379.406385:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114891904:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775768379.406389:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114892032:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775768379.406402:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775768379.406411:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768379/real 1775768379]  req@ffff9f14ab4fdc00 x1862022114892032/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768404 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775768379.406421:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdc00 x1862022114892032/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768404 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775768379.406428:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775768379.406432:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775768379.406435:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114892032:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775768379.406896:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114892160:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775768379.406920:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775768379.406932:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768379/real 1775768379]  req@ffff9f14ab4fce00 x1862022114892160/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768404 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775768379.406944:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fce00 x1862022114892160/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768404 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775768379.406959:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775768379.406963:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775768379.406966:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114892160:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775768379.406972:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114892288:192.168.202.110@tcp:38:kworker.0
00000800:00000100:3.0:1775768379.406992:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775768379.407002:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768379/real 1775768379]  req@ffff9f14ab4ff100 x1862022114892288/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768404 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775768379.407014:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff100 x1862022114892288/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768404 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775768379.407022:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775768379.407025:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775768379.407027:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114892288:192.168.202.110@tcp:38:kworker.0
00000100:00100000:3.0:1775768379.407031:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114892416:192.168.202.110@tcp:8:kworker.0
00000800:00000100:3.0:1775768379.407046:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:3.0:1775768379.407052:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768379/real 1775768379]  req@ffff9f14ab4fc380 x1862022114892416/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768404 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775768379.407059:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fc380 x1862022114892416/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768404 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775768379.407064:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1775768379.407066:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:3.0:1775768379.407069:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114892416:192.168.202.110@tcp:8:kworker.0
00010000:00010000:1.0:1775768379.658877:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768380.426671:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775768380.684735:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768381.450636:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775768381.707518:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768382.474673:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768382.730729:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768383.498742:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775768383.754714:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768384.522973:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775768384.524459:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775768384.524466:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775768384.524470:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768384.524479:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5762
00000100:00080000:3.0:1775768384.524488:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768384.524494:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775768384.524530:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745500 x1862022114892544/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775768384.524581:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775768384.524585:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775768384.524586:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768384.524591:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5762
00000100:00080000:3.0:1775768384.524594:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768384.524597:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775768384.524606:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775768384.524617:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746680 x1862022114892672/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775768384.524626:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775768384.524629:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775768384.524631:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768384.524634:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5762
00000100:00080000:3.0:1775768384.524637:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768384.524641:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775768384.524644:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775768384.524652:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745c00 x1862022114892800/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775768384.524660:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775768384.524663:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775768384.524664:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768384.524668:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5762
00000100:00080000:3.0:1775768384.524671:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768384.524674:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775768384.524678:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775768384.524684:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745180 x1862022114892928/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775768384.524692:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775768384.524695:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775768384.524697:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768384.524700:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5762
00000100:00080000:3.0:1775768384.524702:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768384.524705:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775768384.524709:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775768384.524715:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747100 x1862022114893056/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775768384.524722:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775768384.524726:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775768384.524727:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768384.524730:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5762
00000100:00080000:3.0:1775768384.524733:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768384.524737:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775768384.524740:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775768384.524747:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744000 x1862022114893184/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775768384.524754:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775768384.524757:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775768384.524759:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768384.524762:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5762
00000100:00080000:3.0:1775768384.524764:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768384.524767:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775768384.524770:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775768384.524776:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747800 x1862022114893312/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775768384.524784:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775768384.524786:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775768384.524788:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768384.524791:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5762
00000100:00080000:3.0:1775768384.524794:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768384.524796:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775768384.524801:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775768384.524807:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745f80 x1862022114893440/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775768384.524883:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114892544:192.168.202.110@tcp:250:kworker.0
00000800:00000100:0.0:1775768384.524938:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768384.524971:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768384/real 1775768384]  req@ffff9f1485745500 x1862022114892544/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768409 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768384.524986:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745500 x1862022114892544/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768409 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768384.524996:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768384.525000:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768384.525003:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114892544:192.168.202.110@tcp:250:kworker.0
00000100:00100000:0.0:1775768384.525009:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114892672:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768384.525026:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768384.525035:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768384/real 1775768384]  req@ffff9f1485746680 x1862022114892672/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768409 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768384.525046:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746680 x1862022114892672/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768409 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768384.525053:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768384.525056:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768384.525059:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114892672:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768384.525064:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114892800:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768384.525078:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768384.525085:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768384/real 1775768384]  req@ffff9f1485745c00 x1862022114892800/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768409 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768384.525095:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745c00 x1862022114892800/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768409 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768384.525102:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768384.525105:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768384.525108:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114892800:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768384.525113:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114892928:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768384.525127:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768384.525134:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768384/real 1775768384]  req@ffff9f1485745180 x1862022114892928/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768409 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768384.525143:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745180 x1862022114892928/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768409 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768384.525150:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768384.525153:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768384.525156:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114892928:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775768384.525161:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114893056:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768384.525175:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768384.525183:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768384/real 1775768384]  req@ffff9f1485747100 x1862022114893056/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768409 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768384.525193:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747100 x1862022114893056/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768409 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768384.525200:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768384.525203:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768384.525206:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114893056:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775768384.525210:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114893184:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768384.525223:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768384.525230:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768384/real 1775768384]  req@ffff9f1485744000 x1862022114893184/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768409 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768384.525239:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744000 x1862022114893184/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768409 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768384.525246:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768384.525249:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768384.525252:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114893184:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768384.525256:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114893312:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768384.525269:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768384.525276:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768384/real 1775768384]  req@ffff9f1485747800 x1862022114893312/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768409 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768384.525292:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747800 x1862022114893312/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768409 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768384.525299:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768384.525302:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768384.525305:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114893312:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768384.525309:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114893440:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768384.525322:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768384.525329:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768384/real 1775768384]  req@ffff9f1485745f80 x1862022114893440/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768409 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768384.525339:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745f80 x1862022114893440/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768409 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768384.525346:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768384.525348:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768384.525351:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114893440:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775768384.779155:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768385.546668:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768385.802733:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775768386.574367:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775768386.828450:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768387.594712:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775768387.851410:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768388.618711:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775768388.874726:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1775768389.642662:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768389.642670:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00100000:3.0:1775768389.642673:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:2.0:1775768389.642674:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768389.642685:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5767
00000100:00080000:2.0:1775768389.642697:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768389.642705:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775768389.642744:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2d80 x1862022114893568/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768389.642770:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768389.642773:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775768389.642775:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768389.642779:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5767
00000100:00080000:2.0:1775768389.642783:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768389.642786:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768389.642796:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775768389.642810:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0700 x1862022114893696/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775768389.642812:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114893568:192.168.202.110@tcp:250:kworker.0
00000100:00080000:2.0:1775768389.642827:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768389.642829:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775768389.642831:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768389.642834:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5767
00000100:00080000:2.0:1775768389.642838:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768389.642841:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768389.642846:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775768389.642858:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0380 x1862022114893824/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768389.642867:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768389.642870:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775768389.642871:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768389.642876:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5767
00000800:00000100:0.0:1775768389.642877:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:2.0:1775768389.642882:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768389.642886:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768389.642891:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775768389.642902:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3b80 x1862022114893952/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768389.642910:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768389.642913:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1775768389.642915:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775768389.642927:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768389/real 1775768389]  req@ffff9f14852d2d80 x1862022114893568/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768414 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775768389.642932:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5767
00000100:00080000:2.0:1775768389.642938:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768389.642943:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768389.642949:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775768389.642952:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2d80 x1862022114893568/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768414 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775768389.642964:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3800 x1862022114894080/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775768389.642977:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775768389.642980:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775768389.642985:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:2.0:1775768389.642985:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775768389.642988:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775768389.642992:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114893568:192.168.202.110@tcp:250:kworker.0
00000100:00080000:2.0:1775768389.643007:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5767
00000100:00080000:2.0:1775768389.643012:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768389.643016:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775768389.643020:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114893696:192.168.202.110@tcp:38:kworker.0
00000008:00100000:2.0:1775768389.643023:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775768389.643039:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1f80 x1862022114894208/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1775768389.643050:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:2.0:1775768389.643050:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768389.643055:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775768389.643057:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768389.643062:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5767
00000100:00100000:0.0:1775768389.643067:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768389/real 1775768389]  req@ffff9f14852d0700 x1862022114893696/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768414 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775768389.643073:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768389.643081:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775768389.643085:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0700 x1862022114893696/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768414 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:2.0:1775768389.643087:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00080000:0.0:1775768389.643099:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768389.643104:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775768389.643107:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2680 x1862022114894336/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775768389.643110:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114893696:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768389.643122:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114893824:192.168.202.110@tcp:38:kworker.0
00000100:00080000:2.0:1775768389.643125:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768389.643129:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775768389.643131:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768389.643136:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5767
00000100:00080000:2.0:1775768389.643140:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000800:00000100:0.0:1775768389.643145:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:2.0:1775768389.643149:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768389.643154:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775768389.643160:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768389/real 1775768389]  req@ffff9f14852d0380 x1862022114893824/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768414 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775768389.643168:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2a00 x1862022114894464/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775768389.643178:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0380 x1862022114893824/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768414 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768389.643188:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768389.643191:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768389.643196:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114893824:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768389.643202:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114893952:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768389.643222:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768389.643231:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768389/real 1775768389]  req@ffff9f14852d3b80 x1862022114893952/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768414 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768389.643243:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3b80 x1862022114893952/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768414 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768389.643250:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768389.643253:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768389.643256:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114893952:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775768389.643261:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114894080:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768389.643279:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768389.643289:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768389/real 1775768389]  req@ffff9f14852d3800 x1862022114894080/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768414 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768389.643301:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3800 x1862022114894080/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768414 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768389.643306:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768389.643308:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768389.643310:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114894080:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775768389.643327:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114894208:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768389.643347:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768389.643390:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768389/real 1775768389]  req@ffff9f14852d1f80 x1862022114894208/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768414 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768389.643401:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1f80 x1862022114894208/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768414 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768389.643408:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768389.643411:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768389.643414:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114894208:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768389.643419:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114894336:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768389.643435:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768389.643444:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768389/real 1775768389]  req@ffff9f14852d2680 x1862022114894336/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768414 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768389.643454:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2680 x1862022114894336/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768414 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768389.643461:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768389.643464:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768389.643467:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114894336:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768389.643471:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114894464:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768389.643484:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768389.643491:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768389/real 1775768389]  req@ffff9f14852d2a00 x1862022114894464/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768414 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768389.643985:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2a00 x1862022114894464/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768414 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768389.644001:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768389.644004:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768389.644007:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114894464:192.168.202.110@tcp:8:kworker.0
00010000:00010000:3.0:1775768389.898795:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775768390.666671:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775768390.922802:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775768391.690996:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775768391.946704:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775768392.714834:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768392.971608:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775768393.739383:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775768393.994790:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1775768394.762752:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775768394.762765:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775768394.762770:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768394.762795:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5772
00000100:00080000:3.0:1775768394.762808:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768394.762816:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775768394.762859:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744700 x1862022114894592/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775768394.762956:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114894592:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775768394.763113:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775768394.763117:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775768394.763119:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768394.763125:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5772
00000100:00080000:3.0:1775768394.763130:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768394.763134:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775768394.763144:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775768394.763162:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745f80 x1862022114894720/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:1.0:1775768394.763244:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00100000:0.0:1775768394.763384:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114894720:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775768394.768226:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775768394.768237:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775768394.768242:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768394.768253:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5772
00000100:00080000:3.0:1775768394.768266:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768394.768273:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775768394.768287:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775768394.768343:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747800 x1862022114894848/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775768394.768385:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775768394.768389:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775768394.768391:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768394.768395:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5772
00000100:00080000:3.0:1775768394.768407:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768394.768412:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775768394.768417:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775768394.768429:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744000 x1862022114894976/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775768394.768438:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775768394.768441:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775768394.768443:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768394.768448:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5772
00000100:00080000:3.0:1775768394.768451:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768394.768455:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775768394.768460:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775768394.768469:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747100 x1862022114895104/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775768394.768478:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775768394.768481:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775768394.768483:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768394.768487:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5772
00000100:00080000:3.0:1775768394.768490:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768394.768495:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775768394.768499:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775768394.768511:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745180 x1862022114895232/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775768394.768520:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775768394.768523:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775768394.768524:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768394.768528:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5772
00000100:00080000:3.0:1775768394.768531:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768394.768534:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775768394.768539:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775768394.768547:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745c00 x1862022114895360/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775768394.768576:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775768394.768579:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775768394.768581:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768394.768585:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5772
00000100:00080000:3.0:1775768394.768588:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768394.768591:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775768394.768596:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775768394.768606:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746680 x1862022114895488/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775768394.768859:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114894848:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768394.768944:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114894976:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775768394.768964:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114895104:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775768394.768980:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114895232:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768394.768997:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114895360:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768394.769012:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114895488:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775768395.018776:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775768395.788185:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768396.042800:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775768396.810704:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768397.066787:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775768397.834683:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000400:00000100:2.0:1775768397.835026:0:2392:0:(lib-socket.c:454:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.202.110:988
00000400:00000100:2.0:1775768397.835063:0:2392:0:(acceptor.c:111:lnet_connect_console_error()) Connection to 192.168.202.110@tcp at host 192.168.202.110 was unreachable: the network or that node may be down, or Lustre may be misconfigured.
00000800:00000100:2.0:1775768397.835078:0:2392:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:2.0:1775768397.835119:0:2392:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:2.0:1775768397.835128:0:2392:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:2.0:1775768397.835136:0:2392:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:2.0:1775768397.835158:0:2392:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:2.0:1775768397.835169:0:2392:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:2.0:1775768397.835177:0:2392:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000100:00100000:0.0:1775768397.835185:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768394/real 1775768397]  req@ffff9f1485747800 x1862022114894848/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768419 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000800:00000100:2.0:1775768397.835187:0:2392:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000100:00100000:0.0:1775768397.835217:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747800 x1862022114894848/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768419 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768397.835227:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768397.835233:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768397.835237:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114894848:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768397.835247:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768394/real 1775768397]  req@ffff9f1485744000 x1862022114894976/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768419 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768397.835258:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744000 x1862022114894976/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768419 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768397.835265:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768397.835269:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768397.835296:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114894976:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775768397.835305:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768394/real 1775768397]  req@ffff9f1485747100 x1862022114895104/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768419 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768397.835316:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747100 x1862022114895104/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768419 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768397.835324:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768397.835327:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768397.835330:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114895104:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775768397.835337:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768394/real 1775768397]  req@ffff9f1485745180 x1862022114895232/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768419 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768397.835348:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745180 x1862022114895232/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768419 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768397.835355:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768397.835358:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768397.835361:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114895232:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768397.835368:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768394/real 1775768397]  req@ffff9f1485745c00 x1862022114895360/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768419 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768397.835379:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745c00 x1862022114895360/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768419 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768397.835385:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768397.835389:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768397.835392:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114895360:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768397.835399:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768394/real 1775768397]  req@ffff9f1485746680 x1862022114895488/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768419 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768397.835409:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746680 x1862022114895488/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768419 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768397.835417:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768397.835420:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768397.835423:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114895488:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775768397.835428:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768394/real 1775768397]  req@ffff9f1485745f80 x1862022114894720/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768419 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768397.835438:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745f80 x1862022114894720/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768419 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768397.835446:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768397.835449:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768397.835452:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114894720:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768397.835459:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768394/real 1775768397]  req@ffff9f1485744700 x1862022114894592/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768419 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768397.835470:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744700 x1862022114894592/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768419 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768397.835477:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768397.835481:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768397.835484:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114894592:192.168.202.110@tcp:250:kworker.0
00010000:00010000:1.0:1775768398.090702:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775768398.858697:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775768399.114786:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1775768399.882695:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775768399.882705:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775768399.882709:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00100000:1.0:1775768399.882720:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775768399.882721:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5777
00000100:00080000:3.0:1775768399.882734:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768399.882742:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775768399.882781:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745500 x1862022114895616/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775768399.882811:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775768399.882815:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775768399.882817:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768399.882821:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5777
00000100:00080000:3.0:1775768399.882825:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768399.882829:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775768399.882839:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775768399.882852:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744e00 x1862022114895744/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775768399.882864:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775768399.882867:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775768399.882869:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775768399.882874:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114895616:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775768399.882876:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5777
00000100:00080000:3.0:1775768399.882882:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768399.882886:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775768399.882891:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775768399.882902:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744380 x1862022114895872/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775768399.882910:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775768399.882913:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775768399.882915:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768399.882919:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5777
00000100:00080000:3.0:1775768399.882923:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768399.882926:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775768399.882931:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775768399.882941:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746a00 x1862022114896000/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1775768399.882945:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775768399.882961:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775768399.882964:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775768399.882966:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768399.882970:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5777
00000100:00080000:3.0:1775768399.882974:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768399.882977:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775768399.882982:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775768399.882991:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768399/real 1775768399]  req@ffff9f1485745500 x1862022114895616/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768424 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775768399.882995:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744a80 x1862022114896128/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775768399.883024:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775768399.883027:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775768399.883029:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775768399.883033:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745500 x1862022114895616/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768424 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775768399.883036:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5777
00000100:00080000:3.0:1775768399.883043:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768399.883051:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00080000:0.0:1775768399.883052:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000008:00100000:3.0:1775768399.883056:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00080000:0.0:1775768399.883060:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768399.883065:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114895616:192.168.202.110@tcp:250:kworker.0
00000100:00100000:3.0:1775768399.883071:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747480 x1862022114896256/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775768399.883081:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775768399.883084:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775768399.883086:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768399.883090:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5777
00000100:00100000:0.0:1775768399.883094:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114895744:192.168.202.110@tcp:38:kworker.0
00000100:00080000:3.0:1775768399.883095:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768399.883103:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775768399.883109:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775768399.883120:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745880 x1862022114896384/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775768399.883137:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000800:00000100:0.0:1775768399.883138:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775768399.883142:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775768399.883144:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768399.883150:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5777
00000100:00080000:3.0:1775768399.883155:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1775768399.883158:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768399/real 1775768399]  req@ffff9f1485744e00 x1862022114895744/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768424 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1775768399.883162:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775768399.883170:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775768399.883178:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744e00 x1862022114895744/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768424 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1775768399.883183:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746300 x1862022114896512/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775768399.883198:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768399.883204:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768399.883210:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114895744:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768399.883219:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114895872:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768399.883239:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768399.883249:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768399/real 1775768399]  req@ffff9f1485744380 x1862022114895872/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768424 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768399.883274:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744380 x1862022114895872/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768424 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768399.883281:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768399.883285:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768399.883288:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114895872:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768399.883294:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114896000:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768399.883313:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768399.883323:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768399/real 1775768399]  req@ffff9f1485746a00 x1862022114896000/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768424 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768399.883333:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746a00 x1862022114896000/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768424 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768399.883341:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768399.883344:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768399.883347:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114896000:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775768399.883352:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114896128:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768399.883370:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768399.883379:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768399/real 1775768399]  req@ffff9f1485744a80 x1862022114896128/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768424 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768399.883390:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744a80 x1862022114896128/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768424 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768399.883397:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768399.883400:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768399.883403:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114896128:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775768399.883408:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114896256:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768399.884923:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768399.884958:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768399/real 1775768399]  req@ffff9f1485747480 x1862022114896256/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768424 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768399.884975:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747480 x1862022114896256/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768424 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768399.884984:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768399.884987:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768399.884990:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114896256:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768399.885025:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114896384:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768399.885059:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768399.885069:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768399/real 1775768399]  req@ffff9f1485745880 x1862022114896384/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768424 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768399.885078:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745880 x1862022114896384/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768424 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768399.885085:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768399.885087:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768399.885090:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114896384:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768399.885095:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114896512:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768399.885112:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768399.885121:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768399/real 1775768399]  req@ffff9f1485746300 x1862022114896512/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768424 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768399.885132:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746300 x1862022114896512/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768424 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768399.885139:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768399.885141:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768399.885144:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114896512:192.168.202.110@tcp:8:kworker.0
00010000:00010000:3.0:1775768400.138724:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775768400.906676:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775768401.162720:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775768401.930649:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775768402.187658:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775768402.954681:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768403.210736:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775768403.978690:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768404.235322:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1775768405.002734:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768405.002753:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1775768405.002758:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768405.002766:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5782
00000100:00080000:2.0:1775768405.002782:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768405.002789:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775768405.002838:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0a80 x1862022114896640/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768405.002885:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768405.002889:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775768405.002891:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768405.002896:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5782
00000100:00080000:2.0:1775768405.002900:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768405.002905:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775768405.002913:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114896640:192.168.202.110@tcp:250:kworker.0
00000008:00100000:2.0:1775768405.002915:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775768405.002935:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1500 x1862022114896768/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768405.002943:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768405.002949:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775768405.002950:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768405.002953:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5782
00000100:00080000:2.0:1775768405.002956:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768405.002958:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768405.002961:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775768405.002975:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2a00 x1862022114896896/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1775768405.002976:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:2.0:1775768405.002983:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768405.002986:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775768405.002986:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768405.002989:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5782
00000100:00080000:2.0:1775768405.002991:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768405.002993:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768405.003003:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775768405.003014:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2680 x1862022114897024/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775768405.003019:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768405/real 1775768405]  req@ffff9f14852d0a80 x1862022114896640/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768430 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775768405.003033:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1775768405.003035:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0a80 x1862022114896640/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768430 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775768405.003040:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1775768405.003044:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775768405.003049:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775768405.003057:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5782
00000100:00080000:0.0:1775768405.003060:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:2.0:1775768405.003060:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1775768405.003066:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114896640:192.168.202.110@tcp:250:kworker.0
00000100:00080000:2.0:1775768405.003066:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768405.003078:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775768405.003087:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1f80 x1862022114897152/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775768405.003093:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114896768:192.168.202.110@tcp:38:kworker.0
00000100:00080000:2.0:1775768405.003096:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768405.003103:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775768405.003105:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768405.003108:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5782
00000100:00080000:2.0:1775768405.003112:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768405.003115:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768405.003118:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000800:00000100:0.0:1775768405.003121:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775768405.003131:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3800 x1862022114897280/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775768405.003137:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768405/real 1775768405]  req@ffff9f14852d1500 x1862022114896768/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768430 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775768405.003142:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768405.003149:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775768405.003154:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775768405.003158:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1500 x1862022114896768/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768430 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775768405.003160:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5782
00000100:00080000:2.0:1775768405.003169:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775768405.003170:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768405.003178:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00080000:2.0:1775768405.003178:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775768405.003182:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114896768:192.168.202.110@tcp:38:kworker.0
00000008:00100000:2.0:1775768405.003186:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775768405.003189:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114896896:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775768405.003201:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3b80 x1862022114897408/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:1.0:1775768405.003218:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000800:00000100:0.0:1775768405.003224:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:2.0:1775768405.003232:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768405.003236:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775768405.003238:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775768405.003240:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768405/real 1775768405]  req@ffff9f14852d2a00 x1862022114896896/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768430 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775768405.003244:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5782
00000100:00080000:2.0:1775768405.003250:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768405.003256:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768405.003267:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775768405.003270:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2a00 x1862022114896896/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768430 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768405.003280:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00100000:2.0:1775768405.003281:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0380 x1862022114897536/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775768405.003286:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768405.003293:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114896896:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768405.003305:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114897024:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768405.003333:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768405.003341:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768405/real 1775768405]  req@ffff9f14852d2680 x1862022114897024/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768430 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768405.003352:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2680 x1862022114897024/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768430 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768405.003359:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768405.003362:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768405.003364:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114897024:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775768405.003375:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114897152:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768405.003387:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768405.003393:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768405/real 1775768405]  req@ffff9f14852d1f80 x1862022114897152/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768430 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768405.003400:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1f80 x1862022114897152/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768430 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768405.003406:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768405.003409:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768405.003412:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114897152:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775768405.003417:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114897280:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768405.003441:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768405.003450:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768405/real 1775768405]  req@ffff9f14852d3800 x1862022114897280/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768430 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768405.003461:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3800 x1862022114897280/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768430 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768405.003469:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768405.003471:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768405.003474:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114897280:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768405.003479:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114897408:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768405.003496:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768405.003504:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768405/real 1775768405]  req@ffff9f14852d3b80 x1862022114897408/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768430 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768405.003513:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3b80 x1862022114897408/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768430 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768405.003520:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768405.003522:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768405.003525:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114897408:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768405.003530:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114897536:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768405.003546:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768405.003584:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768405/real 1775768405]  req@ffff9f14852d0380 x1862022114897536/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768430 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768405.003598:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0380 x1862022114897536/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768430 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768405.003611:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768405.003614:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768405.003616:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114897536:192.168.202.110@tcp:8:kworker.0
00010000:00010000:1.0:1775768405.258742:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775768406.026824:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775768406.282754:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775768407.050596:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768407.306729:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775768408.075579:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775768408.330723:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775768409.098631:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775768409.354712:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1775768410.122693:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768410.122701:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00100000:3.0:1775768410.122706:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:2.0:1775768410.122707:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768410.122720:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5787
00000100:00080000:2.0:1775768410.122731:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768410.122738:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775768410.122777:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0700 x1862022114897664/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768410.122804:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768410.122807:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775768410.122809:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768410.122813:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5787
00000100:00080000:2.0:1775768410.122817:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768410.122820:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768410.122829:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775768410.122840:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2d80 x1862022114897792/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768410.122849:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768410.122852:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775768410.122853:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768410.122857:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5787
00000100:00080000:2.0:1775768410.122860:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768410.122864:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768410.122869:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775768410.122881:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0000 x1862022114897920/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775768410.122882:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114897664:192.168.202.110@tcp:250:kworker.0
00000100:00080000:2.0:1775768410.122898:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768410.122901:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775768410.122902:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768410.122906:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5787
00000100:00080000:2.0:1775768410.122910:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768410.122913:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768410.122918:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775768410.122928:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1180 x1862022114898048/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768410.122937:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768410.122940:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1775768410.122941:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768410.122944:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5787
00000100:00080000:2.0:1775768410.122948:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768410.122951:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768410.122955:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:0.0:1775768410.122963:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775768410.122966:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1880 x1862022114898176/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768410.122983:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768410.122986:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775768410.122988:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768410.122991:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5787
00000100:00080000:2.0:1775768410.122995:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768410.122999:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768410.123004:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775768410.123009:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768410/real 1775768410]  req@ffff9f14852d0700 x1862022114897664/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768435 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775768410.123016:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0e00 x1862022114898304/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768410.123032:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1775768410.123037:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0700 x1862022114897664/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768435 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775768410.123037:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775768410.123041:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768410.123050:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5787
00000100:00080000:0.0:1775768410.123056:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775768410.123056:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768410.123063:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00080000:0.0:1775768410.123065:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000008:00100000:2.0:1775768410.123070:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775768410.123071:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114897664:192.168.202.110@tcp:250:kworker.0
00000100:00100000:0.0:1775768410.123082:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114897792:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775768410.123087:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2300 x1862022114898432/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768410.123097:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768410.123100:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775768410.123102:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768410.123106:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5787
00000100:00080000:2.0:1775768410.123111:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000800:00000100:0.0:1775768410.123116:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:2.0:1775768410.123118:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768410.123125:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775768410.123133:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768410/real 1775768410]  req@ffff9f14852d2d80 x1862022114897792/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768435 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775768410.123164:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3100 x1862022114898560/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775768410.123171:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2d80 x1862022114897792/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768435 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768410.123185:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768410.123189:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768410.123193:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114897792:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768410.123215:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114897920:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768410.123234:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768410.123244:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768410/real 1775768410]  req@ffff9f14852d0000 x1862022114897920/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768435 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768410.123256:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0000 x1862022114897920/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768435 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768410.123271:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768410.123274:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768410.123277:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114897920:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768410.123282:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114898048:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768410.123300:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768410.123308:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768410/real 1775768410]  req@ffff9f14852d1180 x1862022114898048/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768435 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768410.123318:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1180 x1862022114898048/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768435 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768410.123325:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768410.123328:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768410.123330:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114898048:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775768410.123335:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114898176:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768410.123350:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768410.123358:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768410/real 1775768410]  req@ffff9f14852d1880 x1862022114898176/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768435 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768410.123368:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1880 x1862022114898176/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768435 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768410.123375:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768410.123378:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768410.123380:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114898176:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775768410.123385:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114898304:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768410.123398:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768410.123407:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768410/real 1775768410]  req@ffff9f14852d0e00 x1862022114898304/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768435 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768410.123417:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0e00 x1862022114898304/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768435 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768410.123424:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768410.123427:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768410.123430:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114898304:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768410.123435:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114898432:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768410.123451:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768410.123459:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768410/real 1775768410]  req@ffff9f14852d2300 x1862022114898432/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768435 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768410.123468:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2300 x1862022114898432/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768435 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768410.123487:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768410.123489:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768410.123492:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114898432:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768410.123497:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114898560:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768410.123512:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768410.123521:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768410/real 1775768410]  req@ffff9f14852d3100 x1862022114898560/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768435 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768410.123531:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3100 x1862022114898560/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768435 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768410.123538:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768410.123541:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768410.123544:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114898560:192.168.202.110@tcp:8:kworker.0
00010000:00010000:3.0:1775768410.378730:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775768411.146682:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768411.402681:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775768412.170660:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775768412.426750:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775768413.194681:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775768413.450697:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775768414.218669:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768414.474728:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1775768415.242657:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768415.242664:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1775768415.242675:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768415.242685:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5792
00000100:00080000:2.0:1775768415.242695:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768415.242702:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775768415.242749:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1c00 x1862022114898688/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768415.242772:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768415.242774:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775768415.242776:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768415.242778:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5792
00000100:00080000:2.0:1775768415.242781:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768415.242784:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768415.242793:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775768415.242807:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3480 x1862022114898816/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:1.0:1775768415.242815:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114898688:192.168.202.110@tcp:250:kworker.0
00000100:00080000:2.0:1775768415.242820:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768415.242824:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775768415.242826:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768415.242831:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5792
00000100:00080000:2.0:1775768415.242836:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768415.242840:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768415.242844:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775768415.242855:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3100 x1862022114898944/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768415.242861:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768415.242864:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775768415.242865:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768415.242867:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5792
00000100:00080000:2.0:1775768415.242870:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768415.242874:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000800:00000100:1.0:1775768415.242876:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000008:00100000:2.0:1775768415.242878:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775768415.242886:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2300 x1862022114899072/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768415.242892:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768415.242895:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1775768415.242896:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768415.242899:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5792
00000100:00080000:2.0:1775768415.242902:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768415.242905:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768415.242910:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775768415.242913:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768415/real 1775768415]  req@ffff9f14852d1c00 x1862022114898688/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768440 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775768415.242922:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0e00 x1862022114899200/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768415.242932:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:1.0:1775768415.242936:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1c00 x1862022114898688/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768440 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775768415.242937:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775768415.242940:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768415.242948:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5792
00000100:00080000:1.0:1775768415.242953:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775768415.242953:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768415.242958:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00080000:1.0:1775768415.242961:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000008:00100000:2.0:1775768415.242964:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775768415.242967:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114898688:192.168.202.110@tcp:250:kworker.0
00000100:00100000:2.0:1775768415.242978:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1880 x1862022114899328/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768415.242987:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768415.242990:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00100000:1.0:1775768415.242992:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114898816:192.168.202.110@tcp:38:kworker.0
00000100:00080000:2.0:1775768415.242993:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768415.243001:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5792
00000100:00080000:2.0:1775768415.243006:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768415.243009:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768415.243015:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000800:00000100:1.0:1775768415.243020:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775768415.243028:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1180 x1862022114899456/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:1.0:1775768415.243035:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768415/real 1775768415]  req@ffff9f14852d3480 x1862022114898816/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768440 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775768415.243042:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768415.243049:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775768415.243053:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768415.243061:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5792
00000100:00080000:2.0:1775768415.243066:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:1.0:1775768415.243070:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3480 x1862022114898816/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768440 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775768415.243071:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768415.243082:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00080000:1.0:1775768415.243083:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775768415.243090:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775768415.243094:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114898816:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775768415.243120:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0000 x1862022114899584/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:1.0:1775768415.243121:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114898944:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775768415.243150:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775768415.243161:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768415/real 1775768415]  req@ffff9f14852d3100 x1862022114898944/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768440 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775768415.243171:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3100 x1862022114898944/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768440 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775768415.243178:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775768415.243182:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775768415.243184:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114898944:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775768415.243190:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114899072:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775768415.243206:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775768415.243214:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768415/real 1775768415]  req@ffff9f14852d2300 x1862022114899072/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768440 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775768415.243224:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2300 x1862022114899072/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768440 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775768415.243236:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775768415.243239:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775768415.243242:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114899072:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775768415.243247:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114899200:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775768415.243262:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775768415.243271:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768415/real 1775768415]  req@ffff9f14852d0e00 x1862022114899200/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768440 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775768415.243280:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0e00 x1862022114899200/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768440 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775768415.243287:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775768415.243290:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775768415.243292:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114899200:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775768415.243297:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114899328:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775768415.243311:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775768415.243319:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768415/real 1775768415]  req@ffff9f14852d1880 x1862022114899328/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768440 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775768415.243328:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1880 x1862022114899328/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768440 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775768415.243335:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775768415.243338:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775768415.243341:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114899328:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775768415.243357:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114899456:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775768415.243371:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775768415.243379:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768415/real 1775768415]  req@ffff9f14852d1180 x1862022114899456/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768440 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775768415.243389:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1180 x1862022114899456/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768440 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775768415.243396:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775768415.243398:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775768415.243401:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114899456:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775768415.243406:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114899584:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775768415.243420:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775768415.243427:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768415/real 1775768415]  req@ffff9f14852d0000 x1862022114899584/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768440 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775768415.243437:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0000 x1862022114899584/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768440 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775768415.243444:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775768415.243447:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775768415.243450:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114899584:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775768415.245266:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775768415.498700:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775768416.266705:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775768416.522809:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775768417.290690:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768417.546895:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775768418.314688:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768418.570793:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775768419.338663:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775768419.594755:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775768420.362718:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775768420.362723:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775768420.362734:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775768420.362738:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768420.362749:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5797
00000100:00080000:3.0:1775768420.362761:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768420.362768:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775768420.362810:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747b80 x1862022114899712/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775768420.362835:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775768420.362838:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775768420.362839:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768420.362842:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5797
00000100:00080000:3.0:1775768420.362844:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768420.362847:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775768420.362855:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775768420.362868:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746300 x1862022114899840/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775768420.362877:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775768420.362882:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775768420.362883:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768420.362887:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5797
00000100:00080000:3.0:1775768420.362889:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768420.362892:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775768420.362895:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775768420.362901:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114899712:192.168.202.110@tcp:250:kworker.0
00000100:00100000:3.0:1775768420.362904:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745880 x1862022114899968/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775768420.362924:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775768420.362926:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775768420.362927:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768420.362930:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5797
00000100:00080000:3.0:1775768420.362932:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768420.362934:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775768420.362936:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775768420.362943:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747480 x1862022114900096/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775768420.362948:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775768420.362949:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775768420.362950:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768420.362952:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5797
00000100:00080000:3.0:1775768420.362954:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768420.362958:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775768420.362960:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775768420.362966:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744a80 x1862022114900224/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775768420.362975:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775768420.362977:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775768420.362978:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768420.362980:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5797
00000100:00080000:3.0:1775768420.362982:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768420.362985:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775768420.362987:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775768420.362994:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746a00 x1862022114900352/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775768420.362999:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775768420.363001:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775768420.363002:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768420.363005:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5797
00000100:00080000:3.0:1775768420.363007:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768420.363009:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775768420.363012:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775768420.363021:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744380 x1862022114900480/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775768420.363034:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775768420.363038:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775768420.363039:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768420.363043:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5797
00000100:00080000:3.0:1775768420.363048:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000800:00000100:1.0:1775768420.363062:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:3.0:1775768420.363070:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775768420.363077:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775768420.363092:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744e00 x1862022114900608/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:1.0:1775768420.363143:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768420/real 1775768420]  req@ffff9f1485747b80 x1862022114899712/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768445 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775768420.363196:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747b80 x1862022114899712/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768445 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775768420.363229:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775768420.363234:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775768420.363238:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114899712:192.168.202.110@tcp:250:kworker.0
00000100:00100000:1.0:1775768420.363277:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114899840:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775768420.363327:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775768420.363369:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768420/real 1775768420]  req@ffff9f1485746300 x1862022114899840/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768445 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775768420.363382:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746300 x1862022114899840/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768445 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775768420.363432:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775768420.363436:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775768420.363459:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114899840:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775768420.363467:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114899968:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775768420.363535:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775768420.363603:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768420/real 1775768420]  req@ffff9f1485745880 x1862022114899968/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768445 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775768420.363617:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745880 x1862022114899968/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768445 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775768420.363624:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775768420.363628:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775768420.363631:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114899968:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775768420.363637:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114900096:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775768420.363686:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775768420.363713:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768420/real 1775768420]  req@ffff9f1485747480 x1862022114900096/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768445 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775768420.363725:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747480 x1862022114900096/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768445 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775768420.363733:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775768420.363736:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775768420.363754:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114900096:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775768420.363778:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114900224:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775768420.363797:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775768420.363837:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768420/real 1775768420]  req@ffff9f1485744a80 x1862022114900224/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768445 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775768420.363868:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744a80 x1862022114900224/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768445 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775768420.363876:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775768420.363879:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775768420.363881:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114900224:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775768420.363887:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114900352:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775768420.363920:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775768420.363931:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768420/real 1775768420]  req@ffff9f1485746a00 x1862022114900352/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768445 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775768420.363959:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746a00 x1862022114900352/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768445 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775768420.363982:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775768420.363986:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775768420.363989:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114900352:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775768420.363995:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114900480:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775768420.364043:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775768420.364079:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768420/real 1775768420]  req@ffff9f1485744380 x1862022114900480/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768445 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775768420.364103:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744380 x1862022114900480/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768445 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775768420.364116:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775768420.364119:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775768420.364122:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114900480:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775768420.364128:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114900608:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775768420.364169:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775768420.364180:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768420/real 1775768420]  req@ffff9f1485744e00 x1862022114900608/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768445 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775768420.364218:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744e00 x1862022114900608/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768445 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775768420.364225:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775768420.364228:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775768420.364231:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114900608:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775768420.618753:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775768421.386683:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768421.642921:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775768422.410701:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775768422.666825:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775768423.434727:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775768423.690784:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775768424.458649:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768424.714754:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1775768425.482712:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:1.0:1775768425.482717:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:0.0:1775768425.482732:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1775768425.482737:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775768425.482747:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5802
00000100:00080000:0.0:1775768425.482761:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775768425.482770:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775768425.482806:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3800 x1862022114900736/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775768425.482831:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775768425.482835:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775768425.482836:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775768425.482841:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5802
00000100:00080000:0.0:1775768425.482845:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775768425.482849:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775768425.482865:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775768425.482880:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3480 x1862022114900864/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:1.0:1775768425.482881:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114900736:192.168.202.110@tcp:250:kworker.0
00000100:00080000:0.0:1775768425.482906:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775768425.482908:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:0.0:1775768425.482909:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775768425.482912:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5802
00000100:00080000:0.0:1775768425.482914:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775768425.482917:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775768425.482919:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775768425.482934:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d2a00 x1862022114900992/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775768425.482943:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775768425.482947:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775768425.482948:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775768425.482952:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5802
00000100:00080000:0.0:1775768425.482970:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775768425.482974:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775768425.482980:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:1.0:1775768425.482989:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768425.483001:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0000 x1862022114901120/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775768425.483034:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775768425.483038:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1775768425.483040:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775768425.483045:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5802
00000100:00080000:0.0:1775768425.483049:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775768425.483055:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775768425.483060:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775768425.483068:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768425/real 1775768425]  req@ffff9f14a04d3800 x1862022114900736/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768450 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768425.483072:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0e00 x1862022114901248/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775768425.483128:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775768425.483134:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775768425.483137:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775768425.483142:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5802
00000100:00080000:0.0:1775768425.483146:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:1.0:1775768425.483148:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3800 x1862022114900736/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768450 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768425.483155:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775768425.483169:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00080000:1.0:1775768425.483173:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00100000:0.0:1775768425.483178:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d1500 x1862022114901376/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775768425.483179:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775768425.483193:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114900736:192.168.202.110@tcp:250:kworker.0
00000100:00080000:0.0:1775768425.483209:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775768425.483212:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:0.0:1775768425.483215:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775768425.483220:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5802
00000100:00080000:0.0:1775768425.483224:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:1.0:1775768425.483224:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114900864:192.168.202.110@tcp:38:kworker.0
00000100:00080000:0.0:1775768425.483241:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775768425.483247:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000800:00000100:1.0:1775768425.483249:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775768425.483261:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768425/real 1775768425]  req@ffff9f14a04d3480 x1862022114900864/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768450 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768425.483269:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3800 x1862022114901504/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:1.0:1775768425.483273:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3480 x1862022114900864/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768450 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775768425.483302:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775768425.483309:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775768425.483320:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114900864:192.168.202.110@tcp:38:kworker.0
00000100:00080000:0.0:1775768425.483324:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:1.0:1775768425.483329:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114900992:192.168.202.110@tcp:38:kworker.0
00000100:00080000:0.0:1775768425.483337:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775768425.483340:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775768425.483345:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5802
00000100:00080000:0.0:1775768425.483352:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775768425.483358:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000800:00000100:1.0:1775768425.483361:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000008:00100000:0.0:1775768425.483366:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775768425.483390:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d2300 x1862022114901632/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:1.0:1775768425.483390:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768425/real 1775768425]  req@ffff9f14a04d2a00 x1862022114900992/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768450 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775768425.483437:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d2a00 x1862022114900992/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768450 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775768425.483452:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775768425.483455:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775768425.483460:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114900992:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775768425.483466:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114901120:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775768425.483483:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775768425.483490:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768425/real 1775768425]  req@ffff9f14a04d0000 x1862022114901120/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768450 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775768425.483497:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0000 x1862022114901120/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768450 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775768425.483502:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775768425.483504:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775768425.483506:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114901120:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775768425.483509:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114901248:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775768425.483522:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775768425.483531:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768425/real 1775768425]  req@ffff9f14a04d0e00 x1862022114901248/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768450 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775768425.483542:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0e00 x1862022114901248/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768450 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775768425.483549:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775768425.483552:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775768425.483574:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114901248:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775768425.483579:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114901376:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775768425.483594:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775768425.483601:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768425/real 1775768425]  req@ffff9f14a04d1500 x1862022114901376/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768450 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775768425.483610:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1500 x1862022114901376/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768450 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775768425.483618:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775768425.483621:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775768425.483624:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114901376:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775768425.483645:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114901504:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775768425.483665:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775768425.483675:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768425/real 1775768425]  req@ffff9f14a04d3800 x1862022114901504/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768450 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775768425.483686:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3800 x1862022114901504/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768450 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775768425.483694:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775768425.483697:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775768425.483700:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114901504:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775768425.483706:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114901632:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775768425.483724:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775768425.483735:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768425/real 1775768425]  req@ffff9f14a04d2300 x1862022114901632/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768450 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775768425.483746:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d2300 x1862022114901632/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768450 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775768425.483754:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775768425.483756:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775768425.483758:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114901632:192.168.202.110@tcp:8:kworker.0
00010000:00010000:3.0:1775768425.738779:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775768426.506720:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768426.762778:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775768427.530702:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775768427.786755:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775768428.554668:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768428.810745:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775768429.578678:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775768429.834722:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775768430.602685:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:2.0:1775768430.603085:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768430.603092:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1775768430.603097:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768430.603118:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5808
00000100:00080000:2.0:1775768430.603131:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768430.603138:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775768430.603176:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2d80 x1862022114901760/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775768430.603253:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114901760:192.168.202.110@tcp:250:kworker.0
00000800:00000100:0.0:1775768430.603313:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:2.0:1775768430.603342:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768430.603346:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775768430.603348:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768430.603352:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5808
00000100:00080000:2.0:1775768430.603356:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768430.603360:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768430.603370:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775768430.603383:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0700 x1862022114901888/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775768430.603493:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768430/real 1775768430]  req@ffff9f14852d2d80 x1862022114901760/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768455 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768430.603505:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2d80 x1862022114901760/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768455 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768430.603512:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768430.603515:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768430.603518:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114901760:192.168.202.110@tcp:250:kworker.0
00000100:00100000:0.0:1775768430.603537:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114901888:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768430.603568:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768430.603578:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768430/real 1775768430]  req@ffff9f14852d0700 x1862022114901888/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768455 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768430.603589:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0700 x1862022114901888/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768455 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768430.603596:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768430.603598:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768430.603602:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114901888:192.168.202.110@tcp:38:kworker.0
00000100:00080000:2.0:1775768430.603672:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768430.603676:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775768430.603677:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768430.603682:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5808
00000100:00080000:2.0:1775768430.603686:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768430.603690:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768430.603695:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775768430.603707:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0380 x1862022114902016/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768430.603717:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768430.603720:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775768430.603722:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768430.603726:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5808
00000100:00080000:2.0:1775768430.603729:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768430.603733:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768430.603738:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775768430.603747:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3b80 x1862022114902144/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768430.603757:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768430.603760:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1775768430.603762:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768430.603765:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5808
00000100:00080000:2.0:1775768430.603768:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768430.603771:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768430.603775:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775768430.603784:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3800 x1862022114902272/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768430.603791:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768430.603794:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775768430.603795:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768430.603798:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5808
00000100:00080000:2.0:1775768430.603800:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768430.603804:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768430.603808:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775768430.603817:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1f80 x1862022114902400/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768430.603824:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768430.603827:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775768430.603828:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768430.603831:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5808
00000100:00080000:2.0:1775768430.603834:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768430.603837:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768430.603841:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775768430.603850:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2680 x1862022114902528/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768430.603858:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768430.603861:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775768430.603862:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768430.603865:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5808
00000100:00080000:2.0:1775768430.603867:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768430.603869:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768430.603873:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775768430.603879:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2a00 x1862022114902656/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775768430.603942:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114902016:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768430.603997:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768430.604008:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768430/real 1775768430]  req@ffff9f14852d0380 x1862022114902016/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768455 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768430.604020:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0380 x1862022114902016/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768455 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768430.604027:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768430.604029:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768430.604032:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114902016:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768430.604038:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114902144:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768430.604058:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768430.604068:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768430/real 1775768430]  req@ffff9f14852d3b80 x1862022114902144/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768455 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768430.604079:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3b80 x1862022114902144/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768455 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768430.604087:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768430.604091:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768430.604094:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114902144:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775768430.604106:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114902272:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768430.604129:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768430.604139:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768430/real 1775768430]  req@ffff9f14852d3800 x1862022114902272/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768455 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768430.604149:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3800 x1862022114902272/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768455 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768430.604154:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768430.604156:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768430.604158:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114902272:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775768430.604162:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114902400:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768430.604176:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768430.604186:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768430/real 1775768430]  req@ffff9f14852d1f80 x1862022114902400/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768455 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768430.604202:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1f80 x1862022114902400/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768455 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768430.604209:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768430.604212:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768430.604214:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114902400:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768430.604219:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114902528:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768430.604234:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768430.604243:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768430/real 1775768430]  req@ffff9f14852d2680 x1862022114902528/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768455 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768430.604252:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2680 x1862022114902528/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768455 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768430.604260:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768430.604262:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768430.604265:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114902528:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768430.604272:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114902656:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768430.604288:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768430.604297:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768430/real 1775768430]  req@ffff9f14852d2a00 x1862022114902656/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768455 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768430.604308:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2a00 x1862022114902656/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768455 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768430.604319:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768430.604321:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768430.604323:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114902656:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775768430.859074:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775768431.626669:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768431.882774:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775768432.650679:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768432.906725:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775768433.674676:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768433.931615:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775768434.698665:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775768434.954747:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775768435.722674:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:0.0:1775768435.724672:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775768435.724677:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1775768435.724680:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775768435.724687:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5813
00000100:00080000:0.0:1775768435.724697:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775768435.724703:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775768435.724739:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0a80 x1862022114902784/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775768435.724765:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775768435.724769:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775768435.724771:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775768435.724774:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5813
00000100:00080000:0.0:1775768435.724778:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775768435.724781:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775768435.724790:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775768435.724802:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0380 x1862022114902912/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775768435.724812:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775768435.724815:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:0.0:1775768435.724817:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775768435.724820:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5813
00000100:00080000:0.0:1775768435.724823:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775768435.724826:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775768435.724831:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775768435.724840:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3100 x1862022114903040/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775768435.724848:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775768435.724851:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775768435.724853:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775768435.724856:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5813
00000100:00080000:0.0:1775768435.724858:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775768435.724862:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775768435.724866:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775768435.724874:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d1180 x1862022114903168/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775768435.724882:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775768435.724885:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1775768435.724886:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775768435.724889:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5813
00000100:00080000:0.0:1775768435.724892:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775768435.724896:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775768435.724900:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775768435.724922:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3b80 x1862022114903296/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:1.0:1775768435.724925:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114902784:192.168.202.110@tcp:250:kworker.0
00000100:00080000:0.0:1775768435.724931:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775768435.724934:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775768435.724935:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775768435.724938:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5813
00000100:00080000:0.0:1775768435.724941:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775768435.724944:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775768435.724948:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775768435.724955:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d1f80 x1862022114903424/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775768435.724962:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775768435.724973:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:0.0:1775768435.724975:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000800:00000100:1.0:1775768435.724979:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:0.0:1775768435.724980:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5813
00000100:00080000:0.0:1775768435.724983:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775768435.724986:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775768435.724990:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775768435.724999:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d2300 x1862022114903552/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775768435.725007:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775768435.725009:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775768435.725011:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:1.0:1775768435.725013:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768435/real 1775768435]  req@ffff9f14a04d0a80 x1862022114902784/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768460 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768435.725014:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5813
00000100:00080000:0.0:1775768435.725017:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775768435.725020:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775768435.725023:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775768435.725028:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0a80 x1862022114902784/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768460 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768435.725031:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3800 x1862022114903680/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775768435.725046:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775768435.725050:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775768435.725054:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114902784:192.168.202.110@tcp:250:kworker.0
00000100:00100000:1.0:1775768435.725060:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114902912:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775768435.725081:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775768435.725091:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768435/real 1775768435]  req@ffff9f14a04d0380 x1862022114902912/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768460 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775768435.725102:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0380 x1862022114902912/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768460 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775768435.725109:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775768435.725112:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775768435.725116:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114902912:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775768435.725121:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114903040:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775768435.725139:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775768435.725149:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768435/real 1775768435]  req@ffff9f14a04d3100 x1862022114903040/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768460 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775768435.725159:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3100 x1862022114903040/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768460 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775768435.725167:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775768435.725169:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775768435.725172:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114903040:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775768435.725177:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114903168:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775768435.725194:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775768435.725204:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768435/real 1775768435]  req@ffff9f14a04d1180 x1862022114903168/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768460 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775768435.725213:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1180 x1862022114903168/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768460 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775768435.725219:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775768435.725222:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775768435.725233:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114903168:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775768435.725256:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114903296:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775768435.725272:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775768435.725280:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768435/real 1775768435]  req@ffff9f14a04d3b80 x1862022114903296/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768460 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775768435.725289:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3b80 x1862022114903296/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768460 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775768435.725296:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775768435.725299:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775768435.725301:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114903296:192.168.202.110@tcp:8:kworker.0
00000100:00100000:1.0:1775768435.725306:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114903424:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775768435.725319:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775768435.725326:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768435/real 1775768435]  req@ffff9f14a04d1f80 x1862022114903424/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768460 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775768435.725354:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1f80 x1862022114903424/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768460 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775768435.725361:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775768435.725363:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775768435.725366:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114903424:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775768435.725371:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114903552:192.168.202.110@tcp:38:kworker.0
00000800:00000100:1.0:1775768435.725386:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775768435.725393:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768435/real 1775768435]  req@ffff9f14a04d2300 x1862022114903552/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768460 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775768435.725402:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d2300 x1862022114903552/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768460 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775768435.725409:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775768435.725411:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775768435.725413:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114903552:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775768435.725419:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114903680:192.168.202.110@tcp:8:kworker.0
00000800:00000100:1.0:1775768435.725433:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775768435.725441:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768435/real 1775768435]  req@ffff9f14a04d3800 x1862022114903680/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768460 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775768435.725451:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3800 x1862022114903680/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768460 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775768435.725463:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1775768435.725466:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:1.0:1775768435.725468:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114903680:192.168.202.110@tcp:8:kworker.0
00010000:00010000:3.0:1775768435.978751:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775768436.746639:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775768437.002756:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775768437.770660:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775768438.026740:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775768438.794658:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768439.050739:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1775768439.818673:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768440.074728:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775768440.842672:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:1.0:1775768440.842672:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775768440.842682:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1775768440.842686:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775768440.842695:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5818
00000100:00080000:1.0:1775768440.842706:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775768440.842714:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775768440.842764:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fca80 x1862022114903808/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775768440.842791:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775768440.842794:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775768440.842795:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775768440.842799:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5818
00000100:00080000:1.0:1775768440.842803:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775768440.842806:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775768440.842816:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775768440.842828:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ffb80 x1862022114903936/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775768440.842830:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114903808:192.168.202.110@tcp:250:kworker.0
00000100:00080000:1.0:1775768440.842845:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775768440.842849:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775768440.842850:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775768440.842854:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5818
00000100:00080000:1.0:1775768440.842869:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775768440.842873:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775768440.842878:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775768440.842889:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fed80 x1862022114904064/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775768440.842897:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775768440.842900:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775768440.842901:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000800:00000100:0.0:1775768440.842906:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:1.0:1775768440.842909:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5818
00000100:00080000:1.0:1775768440.842913:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775768440.842918:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775768440.842923:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775768440.842933:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff800 x1862022114904192/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775768440.842940:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768440/real 1775768440]  req@ffff9f14ab4fca80 x1862022114903808/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768465 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775768440.842950:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775768440.842954:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1775768440.842955:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775768440.842958:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fca80 x1862022114903808/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768465 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775768440.842961:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5818
00000100:00080000:1.0:1775768440.842968:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775768440.842973:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00080000:0.0:1775768440.842974:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000008:00100000:1.0:1775768440.842979:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00080000:0.0:1775768440.842981:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768440.842987:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114903808:192.168.202.110@tcp:250:kworker.0
00000100:00100000:1.0:1775768440.842994:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fc380 x1862022114904320/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775768440.843003:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775768440.843006:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775768440.843008:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775768440.843012:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ffb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114903936:192.168.202.110@tcp:38:kworker.0
00000100:00080000:1.0:1775768440.843016:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5818
00000100:00080000:1.0:1775768440.843022:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775768440.843027:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775768440.843032:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000800:00000100:0.0:1775768440.843038:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775768440.843046:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff100 x1862022114904448/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775768440.843050:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768440/real 1775768440]  req@ffff9f14ab4ffb80 x1862022114903936/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768465 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775768440.843064:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775768440.843071:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775768440.843074:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775768440.843079:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5818
00000100:00100000:0.0:1775768440.843082:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ffb80 x1862022114903936/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768465 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775768440.843084:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775768440.843092:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775768440.843099:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00080000:0.0:1775768440.843101:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768440.843106:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768440.843109:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ffb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114903936:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775768440.843113:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fce00 x1862022114904576/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775768440.843115:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114904064:192.168.202.110@tcp:38:kworker.0
00000100:00080000:1.0:1775768440.843132:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000800:00000100:0.0:1775768440.843133:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:1.0:1775768440.843138:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775768440.843140:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775768440.843145:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5818
00000100:00100000:0.0:1775768440.843148:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768440/real 1775768440]  req@ffff9f14ab4fed80 x1862022114904064/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768465 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775768440.843150:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775768440.843155:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775768440.843161:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775768440.843174:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdc00 x1862022114904704/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775768440.843177:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fed80 x1862022114904064/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768465 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768440.843198:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768440.843202:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768440.843205:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114904064:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768440.843216:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114904192:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768440.843231:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768440.843238:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768440/real 1775768440]  req@ffff9f14ab4ff800 x1862022114904192/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768465 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768440.843245:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff800 x1862022114904192/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768465 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768440.843250:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768440.843251:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768440.843254:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114904192:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775768440.843257:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114904320:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768440.843266:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768440.843270:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768440/real 1775768440]  req@ffff9f14ab4fc380 x1862022114904320/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768465 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768440.843276:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fc380 x1862022114904320/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768465 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768440.843281:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768440.843282:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768440.843284:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114904320:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775768440.843294:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114904448:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768440.843302:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768440.843308:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768440/real 1775768440]  req@ffff9f14ab4ff100 x1862022114904448/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768465 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768440.843317:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff100 x1862022114904448/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768465 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768440.843324:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768440.843327:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768440.843330:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114904448:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768440.843335:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114904576:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768440.843352:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768440.843362:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768440/real 1775768440]  req@ffff9f14ab4fce00 x1862022114904576/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768465 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768440.843373:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fce00 x1862022114904576/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768465 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768440.843381:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768440.843383:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768440.843386:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114904576:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768440.843391:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114904704:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768440.843409:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768440.843419:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768440/real 1775768440]  req@ffff9f14ab4fdc00 x1862022114904704/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768465 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768440.843428:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdc00 x1862022114904704/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768465 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768440.843433:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768440.843434:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768440.843436:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114904704:192.168.202.110@tcp:8:kworker.0
00010000:00010000:3.0:1775768441.098677:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775768441.866667:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775768442.123863:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775768442.890662:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768443.146802:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775768443.914661:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:0.0:1775768444.170904:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775768444.938668:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775768445.197644:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775768445.962669:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:2.0:1775768445.962678:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768445.962685:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1775768445.962688:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768445.962696:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5823
00000100:00080000:2.0:1775768445.962705:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768445.962711:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775768445.962744:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1500 x1862022114904832/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768445.962772:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768445.962774:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775768445.962775:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768445.962778:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5823
00000100:00080000:2.0:1775768445.962780:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768445.962782:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768445.962790:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775768445.962799:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0a80 x1862022114904960/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775768445.962825:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114904832:192.168.202.110@tcp:250:kworker.0
00000100:00080000:2.0:1775768445.962825:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768445.962831:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775768445.962834:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768445.962841:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5823
00000100:00080000:2.0:1775768445.962843:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768445.962845:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768445.962849:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775768445.962861:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2a00 x1862022114905088/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768445.962870:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768445.962873:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775768445.962875:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768445.962878:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5823
00000100:00080000:2.0:1775768445.962882:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768445.962886:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768445.962891:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:0.0:1775768445.962896:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775768445.962905:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2680 x1862022114905216/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768445.962915:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768445.962918:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1775768445.962919:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768445.962923:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5823
00000100:00080000:2.0:1775768445.962926:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768445.962929:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768445.962933:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775768445.962942:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768445/real 1775768445]  req@ffff9f14852d1500 x1862022114904832/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768470 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775768445.962943:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1f80 x1862022114905344/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768445.962962:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768445.962966:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775768445.962967:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768445.962969:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5823
00000100:00080000:2.0:1775768445.962972:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1775768445.962975:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1500 x1862022114904832/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768470 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775768445.962976:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768445.962982:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00080000:0.0:1775768445.962990:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00100000:2.0:1775768445.962993:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3800 x1862022114905472/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775768445.962995:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768445.963001:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114904832:192.168.202.110@tcp:250:kworker.0
00000100:00080000:2.0:1775768445.963014:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768445.963017:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775768445.963019:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775768445.963023:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114904960:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768445.963044:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:2.0:1775768445.963053:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5823
00000100:00100000:0.0:1775768445.963056:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768445/real 1775768445]  req@ffff9f14852d0a80 x1862022114904960/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768470 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775768445.963060:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768445.963068:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768445.963074:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775768445.963075:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0a80 x1862022114904960/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768470 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768445.963086:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00100000:2.0:1775768445.963086:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3b80 x1862022114905600/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775768445.963095:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768445.963100:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114904960:192.168.202.110@tcp:38:kworker.0
00000100:00080000:2.0:1775768445.963100:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768445.963108:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775768445.963109:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1775768445.963111:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114905088:192.168.202.110@tcp:38:kworker.0
00000100:00080000:2.0:1775768445.963114:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5823
00000100:00080000:2.0:1775768445.963118:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768445.963120:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768445.963123:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:0.0:1775768445.963132:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775768445.963136:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0380 x1862022114905728/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775768445.963145:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768445/real 1775768445]  req@ffff9f14852d2a00 x1862022114905088/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768470 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768445.963184:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2a00 x1862022114905088/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768470 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768445.963190:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768445.963193:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768445.963195:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114905088:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768445.963199:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114905216:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768445.963214:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768445.963221:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768445/real 1775768445]  req@ffff9f14852d2680 x1862022114905216/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768470 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768445.963227:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2680 x1862022114905216/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768470 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768445.963232:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768445.963234:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768445.963235:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114905216:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775768445.963238:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114905344:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768445.963246:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768445.963251:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768445/real 1775768445]  req@ffff9f14852d1f80 x1862022114905344/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768470 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768445.963256:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1f80 x1862022114905344/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768470 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768445.963267:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768445.963270:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768445.963273:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114905344:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775768445.963277:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114905472:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768445.963292:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768445.963301:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768445/real 1775768445]  req@ffff9f14852d3800 x1862022114905472/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768470 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768445.963312:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3800 x1862022114905472/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768470 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768445.963319:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768445.963322:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768445.963325:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114905472:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768445.963340:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114905600:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768445.963353:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768445.963359:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768445/real 1775768445]  req@ffff9f14852d3b80 x1862022114905600/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768470 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768445.963365:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3b80 x1862022114905600/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768470 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768445.963370:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768445.963371:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768445.963374:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114905600:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768445.963377:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114905728:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768445.963386:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768445.963390:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768445/real 1775768445]  req@ffff9f14852d0380 x1862022114905728/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768470 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768445.963396:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0380 x1862022114905728/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768470 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768445.963400:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768445.963401:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768445.963403:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114905728:192.168.202.110@tcp:8:kworker.0
00010000:00010000:3.0:1775768446.219875:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775768446.986666:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768447.242722:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775768448.010657:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768448.266983:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775768449.034655:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775768449.290734:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775768450.058657:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768450.314724:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1775768451.082690:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775768451.082699:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1775768451.082711:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775768451.082721:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5828
00000100:00080000:0.0:1775768451.082733:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775768451.082740:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775768451.082789:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d1500 x1862022114905856/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775768451.082814:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:2.0:1775768451.082815:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:0.0:1775768451.082823:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775768451.082825:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775768451.082832:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5828
00000100:00080000:0.0:1775768451.082836:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775768451.082841:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775768451.082851:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775768451.082858:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114905856:192.168.202.110@tcp:250:kworker.0
00000100:00100000:0.0:1775768451.082866:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0e00 x1862022114905984/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:2.0:1775768451.082929:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775768451.082963:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768451/real 1775768451]  req@ffff9f14a04d1500 x1862022114905856/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768476 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775768451.083011:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1500 x1862022114905856/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768476 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775768451.083022:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775768451.083026:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:0.0:1775768451.083046:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775768451.083051:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:0.0:1775768451.083053:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775768451.083058:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5828
00000100:00080000:0.0:1775768451.083062:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775768451.083067:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775768451.083073:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775768451.083085:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0000 x1862022114906112/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775768451.083095:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775768451.083098:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775768451.083100:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775768451.083104:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5828
00000100:00080000:0.0:1775768451.083109:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00100000:2.0:1775768451.083114:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114905856:192.168.202.110@tcp:250:kworker.0
00000100:00080000:0.0:1775768451.083116:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775768451.083125:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775768451.083136:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d2a00 x1862022114906240/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775768451.083146:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:2.0:1775768451.083146:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114905984:192.168.202.110@tcp:38:kworker.0
00000100:00080000:0.0:1775768451.083154:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1775768451.083157:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775768451.083162:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5828
00000100:00080000:0.0:1775768451.083165:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775768451.083168:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775768451.083173:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:2.0:1775768451.083182:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768451.083185:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3480 x1862022114906368/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775768451.083195:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775768451.083198:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775768451.083202:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:2.0:1775768451.083202:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768451/real 1775768451]  req@ffff9f14a04d0e00 x1862022114905984/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768476 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768451.083209:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5828
00000100:00080000:0.0:1775768451.083214:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775768451.083221:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775768451.083225:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775768451.083226:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0e00 x1862022114905984/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768476 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768451.083236:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d1500 x1862022114906496/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768451.083237:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775768451.083246:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00080000:0.0:1775768451.083251:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:2.0:1775768451.083253:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114905984:192.168.202.110@tcp:38:kworker.0
00000100:00080000:0.0:1775768451.083255:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:0.0:1775768451.083259:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775768451.083265:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5828
00000100:00100000:2.0:1775768451.083265:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114906112:192.168.202.110@tcp:38:kworker.0
00000100:00080000:0.0:1775768451.083270:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775768451.083275:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775768451.083279:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775768451.083288:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3800 x1862022114906624/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:2.0:1775768451.083293:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:0.0:1775768451.083301:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775768451.083303:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775768451.083305:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775768451.083308:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5828
00000100:00100000:2.0:1775768451.083311:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768451/real 1775768451]  req@ffff9f14a04d0000 x1862022114906112/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768476 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768451.083313:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775768451.083319:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775768451.083325:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775768451.083334:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0000 x1862022114906112/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768476 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768451.083335:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d2300 x1862022114906752/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768451.083374:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775768451.083379:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775768451.083384:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114906112:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775768451.083403:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114906240:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775768451.083425:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775768451.083440:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768451/real 1775768451]  req@ffff9f14a04d2a00 x1862022114906240/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768476 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775768451.083454:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d2a00 x1862022114906240/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768476 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775768451.083461:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775768451.083465:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775768451.083467:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114906240:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775768451.083473:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114906368:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775768451.083491:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775768451.083500:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768451/real 1775768451]  req@ffff9f14a04d3480 x1862022114906368/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768476 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775768451.083511:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3480 x1862022114906368/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768476 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775768451.083518:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775768451.083521:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775768451.083525:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114906368:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775768451.083530:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114906496:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775768451.083547:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775768451.083575:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768451/real 1775768451]  req@ffff9f14a04d1500 x1862022114906496/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768476 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775768451.083588:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1500 x1862022114906496/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768476 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775768451.083597:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775768451.083600:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775768451.083604:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114906496:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775768451.083611:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114906624:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775768451.083837:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775768451.083852:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768451/real 1775768451]  req@ffff9f14a04d3800 x1862022114906624/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768476 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775768451.083865:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3800 x1862022114906624/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768476 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775768451.083873:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775768451.083876:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775768451.083878:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114906624:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775768451.083884:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114906752:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775768451.083903:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775768451.083913:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768451/real 1775768451]  req@ffff9f14a04d2300 x1862022114906752/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768476 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775768451.083924:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d2300 x1862022114906752/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768476 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775768451.083931:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775768451.083934:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775768451.083937:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114906752:192.168.202.110@tcp:8:kworker.0
00010000:00010000:0.0:1775768451.338692:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768452.106623:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775768452.362797:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768453.130605:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768453.386806:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768454.154662:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768454.410791:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768455.178752:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768455.434763:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768456.202681:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:3.0:1775768456.203358:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775768456.203364:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1775768456.203369:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768456.203378:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5833
00000100:00080000:3.0:1775768456.203389:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768456.203395:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:3.0:1775768456.203429:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745500 x1862022114906880/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775768456.203450:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775768456.203452:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775768456.203454:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768456.203456:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5833
00000100:00080000:3.0:1775768456.203458:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768456.203460:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775768456.203468:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775768456.203476:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744700 x1862022114907008/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775768456.203483:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:2.0:1775768456.203491:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114906880:192.168.202.110@tcp:250:kworker.0
00000100:00080000:3.0:1775768456.203493:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775768456.203496:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768456.203499:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5833
00000100:00080000:3.0:1775768456.203501:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768456.203503:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775768456.203506:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775768456.203512:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745f80 x1862022114907136/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775768456.203517:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775768456.203519:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775768456.203521:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768456.203523:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5833
00000100:00080000:3.0:1775768456.203525:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768456.203526:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775768456.203528:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775768456.203533:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485746680 x1862022114907264/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775768456.203537:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775768456.203539:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1775768456.203540:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768456.203541:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5833
00000100:00080000:3.0:1775768456.203543:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768456.203545:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775768456.203548:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775768456.203577:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745c00 x1862022114907392/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775768456.203582:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775768456.203584:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1775768456.203586:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768456.203588:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5833
00000100:00080000:3.0:1775768456.203590:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768456.203592:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775768456.203595:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775768456.203607:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485745180 x1862022114907520/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775768456.203612:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775768456.203614:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:3.0:1775768456.203615:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768456.203618:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5833
00000100:00080000:3.0:1775768456.203619:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768456.203621:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775768456.203623:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1775768456.203627:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485747100 x1862022114907648/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1775768456.203632:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1775768456.203634:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:3.0:1775768456.203636:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1775768456.203639:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5833
00000100:00080000:3.0:1775768456.203641:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1775768456.203643:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:3.0:1775768456.203645:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:3.0:1775768456.203650:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f1485744000 x1862022114907776/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:2.0:1775768456.203700:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775768456.203753:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768456/real 1775768456]  req@ffff9f1485745500 x1862022114906880/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768481 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775768456.203770:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745500 x1862022114906880/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768481 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775768456.203780:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775768456.203784:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775768456.203788:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114906880:192.168.202.110@tcp:250:kworker.0
00000100:00100000:2.0:1775768456.203809:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114907008:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775768456.203840:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775768456.203852:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768456/real 1775768456]  req@ffff9f1485744700 x1862022114907008/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768481 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775768456.203863:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744700 x1862022114907008/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768481 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775768456.203871:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775768456.203874:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775768456.203877:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114907008:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775768456.203882:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114907136:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775768456.203898:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775768456.203906:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768456/real 1775768456]  req@ffff9f1485745f80 x1862022114907136/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768481 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775768456.203917:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745f80 x1862022114907136/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768481 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775768456.203924:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775768456.203927:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775768456.203930:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114907136:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775768456.203936:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114907264:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775768456.203954:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775768456.203963:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768456/real 1775768456]  req@ffff9f1485746680 x1862022114907264/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768481 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775768456.203980:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485746680 x1862022114907264/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768481 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775768456.203988:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775768456.203991:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775768456.203994:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114907264:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775768456.203999:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114907392:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775768456.204013:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775768456.204021:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768456/real 1775768456]  req@ffff9f1485745c00 x1862022114907392/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768481 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775768456.204031:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745c00 x1862022114907392/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768481 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775768456.204039:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775768456.204042:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775768456.204045:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114907392:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775768456.204050:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114907520:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775768456.204063:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775768456.204072:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768456/real 1775768456]  req@ffff9f1485745180 x1862022114907520/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768481 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775768456.204083:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485745180 x1862022114907520/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768481 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775768456.204096:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775768456.204099:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775768456.204102:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114907520:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775768456.204107:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485747100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114907648:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775768456.204123:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775768456.204131:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768456/real 1775768456]  req@ffff9f1485747100 x1862022114907648/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768481 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775768456.204142:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485747100 x1862022114907648/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768481 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775768456.204149:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775768456.204152:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775768456.204155:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485747100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114907648:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775768456.204160:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114907776:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775768456.204173:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775768456.204255:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768456/real 1775768456]  req@ffff9f1485744000 x1862022114907776/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768481 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775768456.204269:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f1485744000 x1862022114907776/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768481 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775768456.204276:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775768456.204279:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775768456.204283:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f1485744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114907776:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775768456.461981:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768457.226638:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768457.482757:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768458.250656:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:0.0:1775768458.506732:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768459.274660:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775768459.530716:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768460.298695:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768460.554858:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768461.322685:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:2.0:1775768461.325074:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768461.325082:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1775768461.325087:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768461.325098:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5838
00000100:00080000:2.0:1775768461.325109:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768461.325117:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775768461.325159:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0700 x1862022114907904/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768461.325197:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768461.325200:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775768461.325202:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768461.325207:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5838
00000100:00080000:2.0:1775768461.325211:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768461.325215:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768461.325225:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775768461.325238:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2d80 x1862022114908032/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768461.325253:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768461.325256:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775768461.325257:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768461.325261:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5838
00000100:00080000:2.0:1775768461.325264:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768461.325267:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768461.325271:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775768461.325280:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0000 x1862022114908160/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768461.325288:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768461.325291:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775768461.325292:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768461.325295:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5838
00000100:00080000:2.0:1775768461.325298:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768461.325301:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768461.325305:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775768461.325313:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1180 x1862022114908288/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768461.325321:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768461.325324:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1775768461.325325:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768461.325329:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5838
00000100:00080000:2.0:1775768461.325332:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768461.325335:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775768461.325452:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114907904:192.168.202.110@tcp:250:kworker.0
00000008:00100000:2.0:1775768461.325615:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775768461.325630:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1880 x1862022114908416/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768461.325640:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768461.325643:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775768461.325645:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768461.325648:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5838
00000100:00080000:2.0:1775768461.325652:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768461.325656:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768461.325671:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775768461.325682:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0e00 x1862022114908544/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768461.325689:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768461.325692:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775768461.325693:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768461.325696:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5838
00000100:00080000:2.0:1775768461.325699:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768461.325702:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768461.325706:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775768461.325715:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2300 x1862022114908672/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768461.325727:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768461.325729:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775768461.325731:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768461.325734:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5838
00000100:00080000:2.0:1775768461.325737:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768461.325739:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768461.325743:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775768461.325751:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3100 x1862022114908800/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775768461.325846:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114908032:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768461.325874:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114908160:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768461.325892:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114908288:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775768461.325920:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114908416:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775768461.325937:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114908544:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768461.325952:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114908672:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768461.325969:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114908800:192.168.202.110@tcp:8:kworker.0
00010000:00010000:1.0:1775768461.578707:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768462.346690:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768462.603787:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768463.370668:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775768463.626772:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768464.394694:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000400:00000100:3.0:1775768464.394838:0:2393:0:(lib-socket.c:454:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.202.110:988
00000400:00000100:3.0:1775768464.394873:0:2393:0:(acceptor.c:111:lnet_connect_console_error()) Connection to 192.168.202.110@tcp at host 192.168.202.110 was unreachable: the network or that node may be down, or Lustre may be misconfigured.
00000800:00000100:3.0:1775768464.394890:0:2393:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:3.0:1775768464.394925:0:2393:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:3.0:1775768464.394934:0:2393:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:3.0:1775768464.394942:0:2393:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:3.0:1775768464.394951:0:2393:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:3.0:1775768464.394961:0:2393:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000800:00000100:3.0:1775768464.394970:0:2393:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000100:00100000:0.0:1775768464.394973:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768461/real 1775768464]  req@ffff9f14852d1880 x1862022114908416/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768486 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000800:00000100:3.0:1775768464.394980:0:2393:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.110@tcp
00000100:00100000:0.0:1775768464.395002:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1880 x1862022114908416/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768486 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768464.395013:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768464.395018:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768464.395022:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114908416:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775768464.395029:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768461/real 1775768464]  req@ffff9f14852d0e00 x1862022114908544/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768486 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768464.395040:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0e00 x1862022114908544/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768486 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768464.395046:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768464.395048:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768464.395052:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114908544:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768464.395057:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768461/real 1775768464]  req@ffff9f14852d2300 x1862022114908672/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768486 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768464.395074:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2300 x1862022114908672/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768486 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768464.395081:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768464.395084:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768464.395087:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114908672:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768464.395093:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768461/real 1775768464]  req@ffff9f14852d3100 x1862022114908800/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768486 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768464.395103:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3100 x1862022114908800/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768486 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768464.395110:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768464.395113:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768464.395115:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114908800:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775768464.395121:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768461/real 1775768464]  req@ffff9f14852d0700 x1862022114907904/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768486 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768464.395131:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0700 x1862022114907904/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768486 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768464.395139:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768464.395142:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768464.395145:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114907904:192.168.202.110@tcp:250:kworker.0
00000100:00100000:0.0:1775768464.395151:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768461/real 1775768464]  req@ffff9f14852d2d80 x1862022114908032/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768486 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768464.395162:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2d80 x1862022114908032/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768486 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768464.395169:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768464.395171:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768464.395174:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114908032:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768464.395180:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768461/real 1775768464]  req@ffff9f14852d0000 x1862022114908160/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768486 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768464.395188:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0000 x1862022114908160/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768486 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768464.395195:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768464.395198:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768464.395200:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114908160:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768464.395206:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768461/real 1775768464]  req@ffff9f14852d1180 x1862022114908288/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768486 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768464.395216:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1180 x1862022114908288/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768486 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768464.395222:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768464.395225:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768464.395228:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114908288:192.168.202.110@tcp:8:kworker.0
00010000:00010000:3.0:1775768464.651438:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775768465.420074:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:0.0:1775768465.674726:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1775768466.442811:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775768466.442822:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1775768466.442826:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775768466.442837:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5843
00000100:00080000:0.0:1775768466.442848:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775768466.442855:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775768466.442895:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d1f80 x1862022114908928/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1775768466.442916:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:0.0:1775768466.442942:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775768466.442948:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775768466.442950:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775768466.442955:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5843
00000100:00080000:0.0:1775768466.442959:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00100000:2.0:1775768466.442985:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114908928:192.168.202.110@tcp:250:kworker.0
00000800:00000100:2.0:1775768466.443050:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775768466.443094:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768466/real 1775768466]  req@ffff9f14a04d1f80 x1862022114908928/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768491 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775768466.443123:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1f80 x1862022114908928/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768491 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775768466.443134:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775768466.443138:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775768466.443142:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114908928:192.168.202.110@tcp:250:kworker.0
00000100:00080000:0.0:1775768466.443206:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775768466.443217:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775768466.443232:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3b80 x1862022114909056/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775768466.443265:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775768466.443274:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:0.0:1775768466.443276:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775768466.443280:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5843
00000100:00080000:0.0:1775768466.443287:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775768466.443292:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775768466.443298:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775768466.443300:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114909056:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768466.443314:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d1180 x1862022114909184/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775768466.443327:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775768466.443333:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000800:00000100:2.0:1775768466.443333:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:0.0:1775768466.443338:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775768466.443344:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5843
00000100:00080000:0.0:1775768466.443349:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00100000:2.0:1775768466.443353:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768466/real 1775768466]  req@ffff9f14a04d3b80 x1862022114909056/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768491 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768466.443358:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775768466.443367:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775768466.443382:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3100 x1862022114909312/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775768466.443389:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3b80 x1862022114909056/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768491 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768466.443396:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775768466.443404:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1775768466.443405:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768466.443407:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768466.443412:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00080000:0.0:1775768466.443416:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5843
00000100:00100000:2.0:1775768466.443420:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114909056:192.168.202.110@tcp:38:kworker.0
00000100:00080000:0.0:1775768466.443422:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775768466.443430:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775768466.443436:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775768466.443438:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114909184:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768466.443453:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0380 x1862022114909440/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:2.0:1775768466.443463:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:0.0:1775768466.443464:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775768466.443470:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775768466.443472:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:2.0:1775768466.443477:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768466/real 1775768466]  req@ffff9f14a04d1180 x1862022114909184/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768491 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768466.443479:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5843
00000100:00100000:2.0:1775768466.443493:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1180 x1862022114909184/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768491 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768466.443496:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768466.443505:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768466.443506:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00080000:2.0:1775768466.443513:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000008:00100000:0.0:1775768466.443515:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775768466.443522:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114909184:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775768466.443529:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114909312:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775768466.443534:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0a80 x1862022114909568/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775768466.443545:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775768466.443548:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:0.0:1775768466.443551:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000800:00000100:2.0:1775768466.443578:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775768466.443592:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768466/real 1775768466]  req@ffff9f14a04d3100 x1862022114909312/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768491 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768466.443622:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5843
00000100:00080000:0.0:1775768466.443626:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775768466.443630:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775768466.443639:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775768466.443646:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3100 x1862022114909312/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768491 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768466.443654:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3b80 x1862022114909696/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768466.443657:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775768466.443665:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00080000:0.0:1775768466.443671:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:2.0:1775768466.443673:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114909312:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775768466.443691:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114909440:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775768466.443713:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775768466.443723:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768466/real 1775768466]  req@ffff9f14a04d0380 x1862022114909440/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768491 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775768466.443734:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0380 x1862022114909440/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768491 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775768466.443741:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775768466.443744:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775768466.443748:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114909440:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775768466.443752:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114909568:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775768466.443771:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775768466.443781:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768466/real 1775768466]  req@ffff9f14a04d0a80 x1862022114909568/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768491 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775768466.443791:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0a80 x1862022114909568/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768491 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775768466.443804:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775768466.443807:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775768466.443810:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114909568:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775768466.443816:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114909696:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775768466.443834:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775768466.443844:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768466/real 1775768466]  req@ffff9f14a04d3b80 x1862022114909696/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768491 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775768466.443856:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3b80 x1862022114909696/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768491 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775768466.443864:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775768466.443867:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775768466.443869:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114909696:192.168.202.110@tcp:38:kworker.0
00000100:00080000:0.0:1775768466.444787:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775768466.444795:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775768466.444804:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5843
00000100:00080000:0.0:1775768466.444812:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775768466.444817:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775768466.444826:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775768466.444850:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d1f80 x1862022114909824/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775768466.445063:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114909824:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775768466.445268:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775768466.445292:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768466/real 1775768466]  req@ffff9f14a04d1f80 x1862022114909824/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768491 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775768466.445306:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1f80 x1862022114909824/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768491 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775768466.445401:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775768466.445405:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775768466.445409:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114909824:192.168.202.110@tcp:8:kworker.0
00010000:00010000:2.0:1775768466.699991:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775768467.466939:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768467.722933:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775768468.491055:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775768468.746793:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768469.514706:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768469.771971:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768470.538749:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775768470.794766:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1775768471.566671:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768471.566681:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1775768471.566685:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768471.566694:0:74828:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5849
00000100:00080000:2.0:1775768471.566705:0:74828:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768471.566711:0:74828:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775768471.566750:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3480 x1862022114909952/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768471.566784:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768471.566788:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775768471.566789:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768471.566793:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5849
00000100:00080000:2.0:1775768471.566796:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768471.566800:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768471.566810:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775768471.566821:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1c00 x1862022114910080/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768471.566829:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768471.566832:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775768471.566834:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768471.566838:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5849
00000100:00080000:2.0:1775768471.566841:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768471.566844:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768471.566848:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775768471.566858:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d1180 x1862022114910208/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768471.566866:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768471.566868:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775768471.566870:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768471.566873:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5849
00000100:00080000:2.0:1775768471.566876:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768471.566879:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768471.566883:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775768471.566891:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0000 x1862022114910336/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768471.566898:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768471.566901:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1775768471.566902:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768471.566905:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5849
00000100:00080000:2.0:1775768471.566908:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768471.566911:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768471.566915:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775768471.566922:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2d80 x1862022114910464/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768471.566930:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768471.566933:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1775768471.566935:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768471.566938:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5849
00000100:00080000:2.0:1775768471.566941:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768471.566944:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768471.566948:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775768471.566956:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d0700 x1862022114910592/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768471.566964:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768471.566966:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1775768471.566968:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768471.566971:0:74828:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5849
00000100:00080000:2.0:1775768471.566973:0:74828:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768471.566976:0:74828:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768471.566981:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775768471.566988:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d3100 x1862022114910720/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768471.566996:0:74828:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1775768471.566998:0:74828:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:2.0:1775768471.567000:0:74828:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1775768471.567003:0:74828:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5849
00000100:00080000:2.0:1775768471.567006:0:74828:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768471.567009:0:74828:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:2.0:1775768471.567013:0:74828:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775768471.567022:0:74828:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14852d2300 x1862022114910848/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1775768471.569481:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114909952:192.168.202.110@tcp:250:kworker.0
00000800:00000100:0.0:1775768471.569584:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768471.569631:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768471/real 1775768471]  req@ffff9f14852d3480 x1862022114909952/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768496 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768471.569647:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3480 x1862022114909952/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768496 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768471.569657:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768471.569661:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768471.569665:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114909952:192.168.202.110@tcp:250:kworker.0
00000100:00100000:0.0:1775768471.569672:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114910080:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768471.569695:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768471.569706:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768471/real 1775768471]  req@ffff9f14852d1c00 x1862022114910080/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768496 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768471.569716:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1c00 x1862022114910080/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768496 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768471.569724:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768471.569774:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768471.569777:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114910080:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768471.569784:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114910208:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768471.569805:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768471.569816:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768471/real 1775768471]  req@ffff9f14852d1180 x1862022114910208/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768496 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768471.569825:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d1180 x1862022114910208/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768496 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768471.569832:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768471.569835:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768471.569838:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114910208:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768471.569843:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114910336:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768471.569860:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768471.569869:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768471/real 1775768471]  req@ffff9f14852d0000 x1862022114910336/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768496 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768471.569879:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0000 x1862022114910336/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768496 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768471.569885:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768471.569888:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768471.569891:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114910336:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775768471.569896:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114910464:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768471.569914:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768471.569922:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768471/real 1775768471]  req@ffff9f14852d2d80 x1862022114910464/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768496 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768471.569931:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2d80 x1862022114910464/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768496 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768471.569938:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768471.569941:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768471.569944:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114910464:192.168.202.110@tcp:8:kworker.0
00000100:00100000:0.0:1775768471.569950:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114910592:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768471.569967:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768471.569975:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768471/real 1775768471]  req@ffff9f14852d0700 x1862022114910592/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768496 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768471.569984:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d0700 x1862022114910592/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768496 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768471.569992:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768471.569995:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768471.569997:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114910592:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768471.570003:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114910720:192.168.202.110@tcp:38:kworker.0
00000800:00000100:0.0:1775768471.570021:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768471.570030:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768471/real 1775768471]  req@ffff9f14852d3100 x1862022114910720/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768496 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768471.570040:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d3100 x1862022114910720/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768496 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768471.570047:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768471.570050:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768471.570054:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114910720:192.168.202.110@tcp:38:kworker.0
00000100:00100000:0.0:1775768471.570059:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14852d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114910848:192.168.202.110@tcp:8:kworker.0
00000800:00000100:0.0:1775768471.570078:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768471.570087:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768471/real 1775768471]  req@ffff9f14852d2300 x1862022114910848/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768496 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768471.570098:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14852d2300 x1862022114910848/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768496 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768471.570105:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768471.570107:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768471.570110:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14852d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114910848:192.168.202.110@tcp:8:kworker.0
00000100:00100000:3.0:1775768471.572758:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768471.819467:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775768472.586673:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:0.0:1775768472.846025:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775768473.610757:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775768473.866779:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775768474.634728:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768474.894644:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775768475.659841:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775768475.916127:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775768476.682695:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:0.0:1775768476.682748:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775768476.682754:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1775768476.682759:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775768476.682768:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5854
00000100:00080000:0.0:1775768476.682780:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775768476.682786:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775768476.682829:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d2300 x1862022114910976/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775768476.682855:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775768476.682862:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775768476.682863:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775768476.682867:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5854
00000100:00080000:0.0:1775768476.682871:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775768476.682875:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775768476.682886:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775768476.682896:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114910976:192.168.202.110@tcp:250:kworker.0
00000100:00100000:0.0:1775768476.682899:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3800 x1862022114911104/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775768476.682912:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775768476.682915:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:0.0:1775768476.682917:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775768476.682921:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5854
00000100:00080000:0.0:1775768476.682925:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775768476.682928:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775768476.682932:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775768476.682942:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d1500 x1862022114911232/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775768476.682951:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775768476.682955:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775768476.682957:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775768476.682960:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5854
00000100:00080000:0.0:1775768476.682964:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000800:00000100:2.0:1775768476.682966:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:0.0:1775768476.682970:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775768476.682975:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:0.0:1775768476.682985:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3480 x1862022114911360/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775768476.683000:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775768476.683003:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1775768476.683004:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775768476.683007:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5854
00000100:00080000:0.0:1775768476.683010:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775768476.683013:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775768476.683018:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775768476.683018:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768476/real 1775768476]  req@ffff9f14a04d2300 x1862022114910976/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768501 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768476.683032:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d2a00 x1862022114911488/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775768476.683042:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:2.0:1775768476.683042:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d2300 x1862022114910976/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768501 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768476.683051:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775768476.683054:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775768476.683062:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5854
00000100:00080000:2.0:1775768476.683063:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768476.683066:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768476.683070:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:0.0:1775768476.683072:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775768476.683078:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114910976:192.168.202.110@tcp:250:kworker.0
00000008:00100000:0.0:1775768476.683079:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775768476.683094:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0000 x1862022114911616/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775768476.683102:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775768476.683105:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:0.0:1775768476.683106:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775768476.683109:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5854
00000100:00080000:0.0:1775768476.683112:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775768476.683116:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775768476.683120:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114911104:192.168.202.110@tcp:38:kworker.0
00000008:00100000:0.0:1775768476.683121:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775768476.683135:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0e00 x1862022114911744/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775768476.683144:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775768476.683147:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775768476.683149:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000800:00000100:2.0:1775768476.683152:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:0.0:1775768476.683155:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5854
00000100:00080000:0.0:1775768476.683160:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775768476.683163:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775768476.683168:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775768476.683169:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768476/real 1775768476]  req@ffff9f14a04d3800 x1862022114911104/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768501 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768476.683183:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d2300 x1862022114911872/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775768476.683187:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3800 x1862022114911104/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768501 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775768476.683216:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775768476.683220:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775768476.683223:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114911104:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775768476.683228:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114911232:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775768476.683249:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775768476.683258:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768476/real 1775768476]  req@ffff9f14a04d1500 x1862022114911232/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768501 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775768476.683270:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1500 x1862022114911232/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768501 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775768476.683277:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775768476.683279:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775768476.683282:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114911232:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775768476.683287:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114911360:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775768476.683306:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775768476.683315:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768476/real 1775768476]  req@ffff9f14a04d3480 x1862022114911360/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768501 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775768476.683326:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3480 x1862022114911360/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768501 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775768476.683334:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775768476.683337:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775768476.683340:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114911360:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775768476.683346:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114911488:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775768476.683364:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775768476.683374:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768476/real 1775768476]  req@ffff9f14a04d2a00 x1862022114911488/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768501 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775768476.683400:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d2a00 x1862022114911488/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768501 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775768476.683407:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775768476.683410:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775768476.683413:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114911488:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775768476.683418:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114911616:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775768476.683437:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775768476.683447:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768476/real 1775768476]  req@ffff9f14a04d0000 x1862022114911616/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768501 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775768476.683470:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0000 x1862022114911616/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768501 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775768476.683478:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775768476.683481:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775768476.683483:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114911616:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775768476.683503:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114911744:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775768476.683532:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775768476.683542:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768476/real 1775768476]  req@ffff9f14a04d0e00 x1862022114911744/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768501 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775768476.683583:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0e00 x1862022114911744/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768501 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775768476.683591:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775768476.683593:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775768476.683596:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114911744:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775768476.683602:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114911872:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775768476.683620:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775768476.683630:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768476/real 1775768476]  req@ffff9f14a04d2300 x1862022114911872/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768501 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775768476.683641:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d2300 x1862022114911872/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768501 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775768476.683648:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775768476.683650:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775768476.683653:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114911872:192.168.202.110@tcp:8:kworker.0
00010000:00010000:3.0:1775768476.939701:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775768477.706664:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775768477.963240:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1775768478.730693:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768478.987139:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775768479.754706:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:0.0:1775768480.010651:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775768480.778662:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775768481.034714:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775768481.802717:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:1.0:1775768481.802798:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775768481.802805:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1775768481.802810:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775768481.802819:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5859
00000100:00080000:1.0:1775768481.802829:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775768481.802836:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775768481.802870:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fd880 x1862022114912000/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775768481.802893:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775768481.802895:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775768481.802897:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775768481.802900:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5859
00000100:00080000:1.0:1775768481.802903:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775768481.802905:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775768481.802913:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775768481.802922:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fe300 x1862022114912128/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775768481.802932:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775768481.802935:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775768481.802937:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775768481.802942:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5859
00000100:00080000:1.0:1775768481.802945:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775768481.802949:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775768481.802954:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114912000:192.168.202.110@tcp:250:kworker.0
00000008:00100000:1.0:1775768481.802956:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775768481.802969:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdf80 x1862022114912256/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775768481.802977:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775768481.802979:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775768481.802980:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775768481.802982:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5859
00000100:00080000:1.0:1775768481.802985:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775768481.802987:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775768481.802990:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775768481.802998:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fea00 x1862022114912384/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775768481.803003:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775768481.803005:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1775768481.803006:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775768481.803009:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5859
00000100:00080000:1.0:1775768481.803011:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775768481.803013:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775768481.803015:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:2.0:1775768481.803021:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775768481.803024:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdc00 x1862022114912512/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775768481.803030:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775768481.803032:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775768481.803034:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775768481.803036:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5859
00000100:00080000:1.0:1775768481.803039:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775768481.803041:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775768481.803044:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775768481.803052:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fce00 x1862022114912640/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775768481.803060:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775768481.803063:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775768481.803065:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:2.0:1775768481.803067:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768481/real 1775768481]  req@ffff9f14ab4fd880 x1862022114912000/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768506 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775768481.803073:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5859
00000100:00080000:1.0:1775768481.803079:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775768481.803085:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775768481.803090:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775768481.803093:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fd880 x1862022114912000/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768506 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1775768481.803104:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff100 x1862022114912768/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768481.803108:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775768481.803117:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00080000:1.0:1775768481.803119:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775768481.803123:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00100000:2.0:1775768481.803123:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114912000:192.168.202.110@tcp:250:kworker.0
00000100:00080000:1.0:1775768481.803124:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775768481.803132:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5859
00000100:00100000:2.0:1775768481.803136:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114912128:192.168.202.110@tcp:38:kworker.0
00000100:00080000:1.0:1775768481.803138:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775768481.803146:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775768481.803152:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775768481.803164:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fc380 x1862022114912896/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:2.0:1775768481.803168:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775768481.803183:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768481/real 1775768481]  req@ffff9f14ab4fe300 x1862022114912128/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768506 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775768481.803196:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fe300 x1862022114912128/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768506 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775768481.803204:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775768481.803208:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775768481.803211:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114912128:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775768481.803238:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114912256:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775768481.803259:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775768481.803271:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768481/real 1775768481]  req@ffff9f14ab4fdf80 x1862022114912256/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768506 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775768481.803283:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdf80 x1862022114912256/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768506 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775768481.803291:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775768481.803295:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775768481.803299:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114912256:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775768481.803307:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114912384:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775768481.803340:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775768481.803351:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768481/real 1775768481]  req@ffff9f14ab4fea00 x1862022114912384/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768506 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775768481.803363:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fea00 x1862022114912384/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768506 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775768481.803371:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775768481.803374:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775768481.803377:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114912384:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775768481.803382:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114912512:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775768481.803401:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775768481.803411:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768481/real 1775768481]  req@ffff9f14ab4fdc00 x1862022114912512/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768506 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775768481.803422:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdc00 x1862022114912512/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768506 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775768481.803429:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775768481.803432:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775768481.803435:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114912512:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775768481.803441:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114912640:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775768481.803490:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775768481.803501:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768481/real 1775768481]  req@ffff9f14ab4fce00 x1862022114912640/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768506 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775768481.803513:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fce00 x1862022114912640/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768506 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775768481.803521:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775768481.803523:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775768481.803527:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114912640:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775768481.803532:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114912768:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775768481.803593:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775768481.803605:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768481/real 1775768481]  req@ffff9f14ab4ff100 x1862022114912768/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768506 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775768481.803616:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff100 x1862022114912768/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768506 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775768481.803623:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775768481.803626:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775768481.803629:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114912768:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775768481.803634:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114912896:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775768481.803653:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775768481.803662:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768481/real 1775768481]  req@ffff9f14ab4fc380 x1862022114912896/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768506 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775768481.803672:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fc380 x1862022114912896/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768506 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775768481.803699:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775768481.803703:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775768481.803706:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114912896:192.168.202.110@tcp:8:kworker.0
00010000:00010000:3.0:1775768482.058730:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775768482.826682:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775768483.082797:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775768483.850712:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768484.106754:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775768484.874729:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775768485.130675:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775768485.898684:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768486.154732:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775768486.922659:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:1.0:1775768486.922661:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775768486.922671:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1775768486.922676:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775768486.922686:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5864
00000100:00080000:1.0:1775768486.922698:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775768486.922706:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:1.0:1775768486.922742:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff800 x1862022114913024/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775768486.922763:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775768486.922766:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775768486.922767:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775768486.922770:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5864
00000100:00080000:1.0:1775768486.922772:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775768486.922775:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775768486.922783:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775768486.922791:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fed80 x1862022114913152/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775768486.922794:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114913024:192.168.202.110@tcp:250:kworker.0
00000100:00080000:1.0:1775768486.922804:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775768486.922806:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775768486.922807:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775768486.922810:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5864
00000100:00080000:1.0:1775768486.922811:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775768486.922813:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775768486.922816:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1775768486.922824:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ffb80 x1862022114913280/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775768486.922829:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775768486.922831:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775768486.922832:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775768486.922834:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5864
00000100:00080000:1.0:1775768486.922837:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775768486.922839:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775768486.922842:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775768486.922848:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fca80 x1862022114913408/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775768486.922853:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775768486.922856:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1775768486.922859:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000800:00000100:2.0:1775768486.922861:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00080000:1.0:1775768486.922867:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5864
00000100:00080000:1.0:1775768486.922871:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775768486.922875:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775768486.922879:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:1.0:1775768486.922891:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fc380 x1862022114913536/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1775768486.922900:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:2.0:1775768486.922910:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768486/real 1775768486]  req@ffff9f14ab4ff800 x1862022114913024/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768511 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775768486.922914:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1775768486.922917:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775768486.922924:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5864
00000100:00080000:1.0:1775768486.922927:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775768486.922931:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:2.0:1775768486.922934:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff800 x1862022114913024/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768511 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:1.0:1775768486.922935:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00080000:2.0:1775768486.922946:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00100000:1.0:1775768486.922950:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4ff100 x1862022114913664/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1775768486.922953:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775768486.922959:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114913024:192.168.202.110@tcp:250:kworker.0
00000100:00080000:1.0:1775768486.922972:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775768486.922975:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:1.0:1775768486.922977:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775768486.922982:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5864
00000100:00100000:2.0:1775768486.922985:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114913152:192.168.202.110@tcp:38:kworker.0
00000100:00080000:1.0:1775768486.922987:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1775768486.922994:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775768486.922999:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000800:00000100:2.0:1775768486.923008:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:1.0:1775768486.923014:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fce00 x1862022114913792/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775768486.923019:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768486/real 1775768486]  req@ffff9f14ab4fed80 x1862022114913152/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768511 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775768486.923027:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1775768486.923033:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:1.0:1775768486.923036:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1775768486.923042:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5864
00000100:00080000:1.0:1775768486.923046:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00100000:2.0:1775768486.923047:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fed80 x1862022114913152/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768511 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1775768486.923050:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:1.0:1775768486.923058:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00080000:2.0:1775768486.923059:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775768486.923063:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775768486.923066:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114913152:192.168.202.110@tcp:38:kworker.0
00000100:00100000:1.0:1775768486.923069:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14ab4fdc00 x1862022114913920/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775768486.923075:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ffb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114913280:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775768486.923091:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775768486.923097:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768486/real 1775768486]  req@ffff9f14ab4ffb80 x1862022114913280/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768511 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775768486.923108:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ffb80 x1862022114913280/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768511 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775768486.923116:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775768486.923119:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775768486.923122:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ffb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114913280:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775768486.923139:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114913408:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775768486.923164:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775768486.923175:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768486/real 1775768486]  req@ffff9f14ab4fca80 x1862022114913408/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768511 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775768486.923189:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fca80 x1862022114913408/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768511 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775768486.923206:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775768486.923209:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775768486.923211:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114913408:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775768486.923217:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114913536:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775768486.923234:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775768486.923243:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768486/real 1775768486]  req@ffff9f14ab4fc380 x1862022114913536/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768511 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775768486.923252:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fc380 x1862022114913536/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768511 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775768486.923257:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775768486.923260:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775768486.923261:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114913536:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775768486.923266:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114913664:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775768486.923281:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775768486.923304:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768486/real 1775768486]  req@ffff9f14ab4ff100 x1862022114913664/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768511 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775768486.923316:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4ff100 x1862022114913664/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768511 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775768486.923324:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775768486.923328:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775768486.923331:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114913664:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775768486.923349:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114913792:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775768486.923365:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775768486.923375:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768486/real 1775768486]  req@ffff9f14ab4fce00 x1862022114913792/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768511 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775768486.923384:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fce00 x1862022114913792/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768511 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775768486.923393:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775768486.923395:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775768486.923396:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114913792:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775768486.923399:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114913920:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775768486.923435:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775768486.923445:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768486/real 1775768486]  req@ffff9f14ab4fdc00 x1862022114913920/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768511 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775768486.923464:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14ab4fdc00 x1862022114913920/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768511 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775768486.923471:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775768486.923475:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775768486.923477:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14ab4fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114913920:192.168.202.110@tcp:8:kworker.0
00010000:00010000:0.0:1775768487.178713:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775768487.946652:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775768488.202694:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775768488.970648:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768489.226741:0:60132:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775768489.994656:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:1.0:1775768490.250706:0:60134:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:0.0:1775768491.018663:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768491.274707:0:60133:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1775768492.042622:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775768492.042631:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00100000:2.0:1775768492.042632:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00000100:00080000:0.0:1775768492.042635:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775768492.042644:0:37:0:(import.c:509:import_select_connection()) MGC192.168.202.110@tcp: connect to NID 192.168.202.110@tcp last attempt 5869
00000100:00080000:0.0:1775768492.042654:0:37:0:(import.c:588:import_select_connection()) MGC192.168.202.110@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775768492.042667:0:37:0:(import.c:623:import_select_connection()) MGC192.168.202.110@tcp: import ffff9f1494a33800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000100:00100000:0.0:1775768492.042702:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d1f80 x1862022114914048/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775768492.042725:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775768492.042727:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775768492.042729:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775768492.042731:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5869
00000100:00080000:0.0:1775768492.042734:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775768492.042737:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14902b3800: import ffff9f14907d2000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775768492.042745:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1775768492.042750:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114914048:192.168.202.110@tcp:250:kworker.0
00000100:00100000:0.0:1775768492.042754:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3b80 x1862022114914176/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775768492.042761:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775768492.042763:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:0.0:1775768492.042764:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775768492.042766:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5869
00000100:00080000:0.0:1775768492.042768:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775768492.042770:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14902b3800: import ffff9f14907d3000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775768492.042773:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775768492.042781:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0a80 x1862022114914304/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775768492.042786:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775768492.042789:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775768492.042790:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775768492.042792:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5869
00000100:00080000:0.0:1775768492.042794:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775768492.042796:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14902b3800: import ffff9f14907d4800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775768492.042799:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:2.0:1775768492.042803:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768492.042811:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d0380 x1862022114914432/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775768492.042818:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775768492.042820:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1775768492.042821:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775768492.042823:0:37:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: connect to NID 192.168.202.110@tcp last attempt 5869
00000100:00080000:0.0:1775768492.042825:0:37:0:(import.c:588:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775768492.042827:0:37:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff9f14902b3800: import ffff9f14907d5800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775768492.042830:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000100:00100000:2.0:1775768492.042836:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768492/real 1775768492]  req@ffff9f14a04d1f80 x1862022114914048/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768517 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1775768492.042840:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d3100 x1862022114914560/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1775768492.042857:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775768492.042859:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1775768492.042861:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:2.0:1775768492.042863:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1f80 x1862022114914048/t0(0) o250->MGC192.168.202.110@tcp@192.168.202.110@tcp:26/25 lens 520/544 e 0 to 1 dl 1775768517 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1775768492.042870:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5869
00000100:00080000:0.0:1775768492.042874:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1775768492.042876:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f1494a33800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1775768492.042877:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff9f14b38c6800: import ffff9f14ab61b000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775768492.042882:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00080000:2.0:1775768492.042886:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.110@tcp failed (-110)
00000100:00100000:0.0:1775768492.042890:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d1180 x1862022114914688/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775768492.042893:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7fce0e0a-bb6e-4448-8ea1-2b470d3cbc41:2402:1862022114914048:192.168.202.110@tcp:250:kworker.0
00000100:00080000:0.0:1775768492.042902:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775768492.042904:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:0.0:1775768492.042905:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775768492.042908:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5869
00000100:00080000:0.0:1775768492.042909:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775768492.042911:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff9f14b38c6800: import ffff9f14ab619800 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775768492.042914:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681677 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1775768492.042919:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d1f80 x1862022114914816/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775768492.042925:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114914176:192.168.202.110@tcp:38:kworker.0
00000100:00080000:0.0:1775768492.042927:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) f7291b10-c63d-4c01-a1d1-66463b7d0b42->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1775768492.042932:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery
00000100:00080000:0.0:1775768492.042934:0:37:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1775768492.042937:0:37:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: connect to NID 192.168.202.110@tcp last attempt 5869
00000100:00080000:0.0:1775768492.042939:0:37:0:(import.c:588:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1775768492.042941:0:37:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff9f14b38c6800: import ffff9f14ab618000 using connection 192.168.202.110@tcp/192.168.202.110@tcp
00000008:00100000:0.0:1775768492.042947:0:37:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34681677 ocd_grant: 8388608, lost: 0.
00000800:00000100:2.0:1775768492.042955:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:0.0:1775768492.042957:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff9f14a04d2300 x1862022114914944/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1775768492.042969:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768492/real 1775768492]  req@ffff9f14a04d3b80 x1862022114914176/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768517 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775768492.042978:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3b80 x1862022114914176/t0(0) o38->lustre-MDT0000-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768517 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775768492.042983:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d2000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775768492.042985:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775768492.042987:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114914176:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775768492.042990:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114914304:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775768492.043001:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775768492.043006:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768492/real 1775768492]  req@ffff9f14a04d0a80 x1862022114914304/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768517 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775768492.043012:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0a80 x1862022114914304/t0(0) o38->lustre-MDT0001-mdc-ffff9f14902b3800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768517 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775768492.043016:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d3000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775768492.043018:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775768492.043019:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114914304:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775768492.043023:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114914432:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775768492.043031:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775768492.043036:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768492/real 1775768492]  req@ffff9f14a04d0380 x1862022114914432/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768517 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775768492.043042:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d0380 x1862022114914432/t0(0) o8->lustre-OST0000-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768517 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775768492.043046:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d4800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775768492.043048:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775768492.043049:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114914432:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775768492.043052:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114914560:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775768492.043060:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775768492.043064:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768492/real 1775768492]  req@ffff9f14a04d3100 x1862022114914560/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768517 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775768492.043070:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d3100 x1862022114914560/t0(0) o8->lustre-OST0001-osc-ffff9f14902b3800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768517 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775768492.043074:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14907d5800 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775768492.043076:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775768492.043077:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5ca127d7-bcfc-4ef2-8b1d-a4c9b5152e48:2402:1862022114914560:192.168.202.110@tcp:8:kworker.0
00000100:00100000:2.0:1775768492.043080:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114914688:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775768492.043087:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775768492.043092:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768492/real 1775768492]  req@ffff9f14a04d1180 x1862022114914688/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768517 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775768492.043102:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1180 x1862022114914688/t0(0) o38->lustre-MDT0000-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768517 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775768492.043106:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab61b000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775768492.043107:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775768492.043109:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114914688:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775768492.043119:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114914816:192.168.202.110@tcp:38:kworker.0
00000800:00000100:2.0:1775768492.043128:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775768492.043132:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768492/real 1775768492]  req@ffff9f14a04d1f80 x1862022114914816/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768517 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775768492.043151:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d1f80 x1862022114914816/t0(0) o38->lustre-MDT0001-mdc-ffff9f14b38c6800@192.168.202.110@tcp:12/10 lens 520/544 e 0 to 1 dl 1775768517 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775768492.043155:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab619800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775768492.043157:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775768492.043159:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114914816:192.168.202.110@tcp:38:kworker.0
00000100:00100000:2.0:1775768492.043162:0:2402:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9f14a04d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114914944:192.168.202.110@tcp:8:kworker.0
00000800:00000100:2.0:1775768492.043175:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.110@tcp
00000100:00100000:2.0:1775768492.043181:0:2402:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1775768492/real 1775768492]  req@ffff9f14a04d2300 x1862022114914944/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768517 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1775768492.043187:0:2402:0:(client.c:2510:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff9f14a04d2300 x1862022114914944/t0(0) o8->lustre-OST0000-osc-ffff9f14b38c6800@192.168.202.110@tcp:28/4 lens 520/544 e 0 to 1 dl 1775768517 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1775768492.043195:0:2402:0:(import.c:65:import_set_state_nolock()) ffff9f14ab618000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1775768492.043197:0:2402:0:(import.c:1506:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.110@tcp failed (-110)
00000100:00100000:2.0:1775768492.043198:0:2402:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9f14a04d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f7291b10-c63d-4c01-a1d1-66463b7d0b42:2402:1862022114914944:192.168.202.110@tcp:8:kworker.0
00010000:00010000:1.0:1775768492.298727:0:39314:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768493.066669:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:3.0:1775768493.322709:0:57277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768494.090615:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:0.0:1775768494.346648:0:39315:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768495.114675:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
00010000:00010000:2.0:1775768495.370723:0:46896:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1775768496.138630:0:73979:0:(client.c:2691:ptlrpc_set_wait()) set ffff9f14a0716480 going to sleep for 0 seconds
Debug log: 77007 lines, 77007 kept, 0 dropped, 0 bad.